A three-dimensional memory device includes an alternating stack of insulating layers and electrically conductive layers, where the electrically conductive layers include word-line-level electrically conductive layers and drain-select-level electrically conductive layers located above the word-line-level electrically conductive layers, memory opening fill structures vertically extending through the alternating stack, and drain-select-level contact via structures. A first one of the drain-select level contact structures directly contacts at least a first two of the drain-select-level electrically conductive layers that are vertically spaced apart from each other. A second one of the drain-select level contact structures directly contacts at least a second two of the drain-select-level electrically conductive layers that are vertically spaced apart from each other and which are located below the at least the first two of the drain-select-level electrically conductive layers.