A three-dimensional memory device includes an alternating stack of insulating layers and electrically conductive layers located over a substrate, the alternating stack containing a memory array region and a terrace region. Memory stack structures containing a memory film and a vertical semiconductor channel extend through the memory array region of the alternating stack. Support pillar structures extending through the terrace region of the alternating stack. The support pillar structures have different heights from each other.