An alternating stack of insulating layers and sacrificial material layers is formed over a substrate. Memory stack structures are formed through the alternating stack. Drain-select-level trenches through an upper subset of the sacrificial material layers, and backside trenches are formed through each layer of the alternating stack. Backside recesses are formed by removing the sacrificial material layers. A first electrically conductive material and a second electrically conductive material are sequentially deposited in the backside recesses and the drain-select-level trenches. Portions of the second electrically conductive material and the first electrically conductive material may be removed by at least one anisotropic etch process from the drain-select-level trenches to provide drain-select-level electrically conductive layers as multiple groups that are laterally spaced apart and electrically isolated from one another by cavities within the drain-select-level trenches.