A memory device includes first conductive rails laterally extending along a first horizontal direction over a substrate, a rectangular array of first memory pillar structures, each containing a memory element, overlying top surfaces of the first conductive rails, second conductive rails laterally extending along a second horizontal direction and overlying top surfaces of the rectangular array of first memory pillar structures, and a one-dimensional array of first cavities free of solid material portions therein, laterally extending along the second horizontal direction and located between neighboring pairs of the second conductive rails.