A three-dimensional memory device includes an alternating stack of insulating layers and electrically conductive layers located over a substrate, and memory pillar structures extending through the alternating stack. Each of the memory pillar structures includes a respective memory film and a respective vertical semiconductor channel. Dielectric cores contact an inner sidewall of a respective one of the vertical semiconductor channels. A drain-select-level isolation structure laterally extends along a first horizontal direction and contacts straight sidewalls of the dielectric cores at a respective two-dimensional flat interface. The memory pillar structures may be formed on-pitch as a two-dimensional periodic array, and themay drain-select-level isolation structure may cut through upper portions of the memory pillar structures to minimize areas occupied by the drain-select-level isolation structure.