A three-dimensional memory device includes an alternating stack of insulating layers and electrically conductive layers, memory opening fill structures including a respective vertical semiconductor channel and a respective vertical stack of memory elements extending through the alternating stack in a memory array region, via contact structures contacting the stepped surfaces of the electrically conductive layers at each step in a staircase region, and a vertical stack of access transistors located between the staircase region and the memory array region.