An annular dielectric spacer can be formed at a level of a joint-level dielectric material layer between vertically neighboring pairs of alternating stacks of insulating layers and spacer material layers. After formation of a memory opening through multiple alternating stacks and formation of a memory film therein, an anisotropic etch can be performed to remove a horizontal bottom portion of the memory film. The annular dielectric spacer can protect underlying portions of the memory film during the anisotropic etch. In addition, a silicon nitride barrier may be employed to suppress hydrogen diffusion at an edge region of peripheral devices.