A three-dimensional memory device includes alternating stacks of insulating layers and electrically conductive layers. The alternating stacks are laterally spaced apart among one another by backside isolation assemblies. At least one of the backside isolation assemblies generally extends along a first horizontal direction with lateral undulations along a second horizontal direction that is perpendicular to the first horizontal direction. At least one of the alternating stacks has a modulation in width along the second horizontal direction as a function of a position along the first horizontal direction. Memory stack structures vertically extend through a respective one of the alternating stacks. Each of the backside isolation assemblies includes a respective laterally alternating sequence of backside dielectric isolation walls and backside dielectric support pillar structures.