A three-dimensional memory device includes a plurality of alternating stacks of insulating layers and electrically conductive layers located over a substrate, clusters of memory stack structures vertically extending through a respective one of the alternating stacks, and bit lines electrically connected to an upper end of a respective subset of the vertical semiconductor channels. In one embodiment, a subset of the bit lines can include a respective multi-level structure. Each multi-level structure includes bit-line-level bit line segments and an interconnection line segment located at a different level from the bit-line-level bit line segments. In another embodiment, groups of alternating stacks can be alternately indented along a horizontal direction perpendicular to the bit lines to provide dielectric material portions located in lateral indentation regions. Metal line structures connecting contact via structures can extend parallel to bit lines to provide electrical connections between word lines and underlying field effect transistors.