An alternating stack of insulating layers and spacer material layers is formed over a substrate. An insulating cap layer is formed thereupon. A memory opening is formed, which has a greater lateral dimension at a level of an upper insulating cap sublayer than at a level of a lower insulating cap sublayer. A memory film and a semiconductor channel material layer is formed in the memory opening. Ions of at least one dopant species is implanted into a top portion of the semiconductor channel material layer. An isotropic etch process etches an unimplanted portion of the semiconductor channel material layer at a higher etch rate than the implanted top portion of the semiconductor channel material layer to form a vertical semiconductor channel.