An alternating stack of insulating layers including a silicon oxide material and electrically conductive layers is formed over a substrate. Sidewalls of the insulating layers are selectively silylated with a chemical including at least one silyl group without silylating sidewalls of the electrically conductive layers. Silicon-containing barrier material portions are formed by selectively growing a first silicon-containing barrier material from surfaces of the electrically conductive layers without growing the first silicon-containing barrier material from silylated surfaces of the insulating layers. A memory material layer is formed on the silicon-containing barrier material portions and the sidewalls of the insulating layers. A vertical conductive line is formed on the memory material layer.