A lower source-level dielectric etch-stop layer, a source-level sacrificial layer, and an upper source-level dielectric etch-stop layer are formed over a substrate. An alternating stack of insulating layers and sacrificial material layers is formed thereabove. Memory stack structures are formed through the alternating stack. Backside openings are formed through the alternating stack and into the in-process source-level material layers such that tapered surfaces are formed through the upper source-level dielectric etch-stop layer. A source cavity is formed by removing the source-level sacrificial layer, and a continuous source contact layer is formed in the source cavity and in peripheral portions of the backside openings. Portions of the continuous source contact layer overlying the tapered surfaces are removed by performing an isotropic etch process. Remaining portions of the continuous source contact layer comprise a source contact layer. The sacrificial material layers are replaced with electrically conductive layers.