An alternating stack of insulating layers and sacrificial material layers is formed over a substrate. The sacrificial material layers are formed as, or are subsequently replaced with, electrically conductive layers. Memory openings are formed through the alternating stack. A memory film is formed within each memory openings. A silicon-germanium alloy layer including germanium at an atomic concentration less than 25% is deposited within each memory opening. An oxidation process is performed on the silicon-germanium alloy layer. A vertical semiconductor channel including an unoxidized remaining material portion of the silicon-germanium alloy layer is formed, which includes germanium at an atomic concentration greater than 50%.