Memory stack structures and dielectric wall structures are formed through a vertically alternating sequence of continuous insulating layers and continuous sacrificial material layers. Backside trenches are formed to divide the vertically alternating sequence into multiple alternating stacks. First portions of the continuous sacrificial material layers are replaced with electrically conductive layers. A connection region including a pair of dielectric wall structures is provided between a first memory array region and a second memory array region of a first alternating stack. Second portions of the continuous sacrificial material layers remain between the pair of dielectric wall structures as a vertical stack of dielectric plates. An upper subset of the first electrically conductive layers is patterned and is divided into multiple discrete portions. The multiple discrete portions are electrically connected by a respective set of connection metal interconnect structures. A metal via structure may be formed through the dielectric plates.