A three-dimensional memory device includes an alternating stack of insulating layers and electrically conductive layers located over a substrate, memory stack structures vertically extending through the alternating stack, a finned dielectric moat structure including a dielectric core portion vertically extending through each layer within the alternating stack and a vertical stack of dielectric fin portions laterally extending outward from the dielectric core portion, a vertical stack of insulating plates and dielectric material plates laterally surrounded by the finned dielectric moat structure, and an interconnection via structure vertically extending through the vertical stack and contacting a top surface of an underlying metal interconnect structure.