Air gaps are provided to reduce interference and resistance between metal bit lines in non-volatile memory structures. Metal vias can be formed that are electrically coupled with the drain region of an underlying device and extend vertically with respect to the substrate surface to provide contacts for bit lines that are elongated in a column direction above. The metal vias can be separated by a dielectric fill material. Layer stack columns extend in a column direction over the dielectric fill and metal vias. Each layer stack column includes a metal bit line over a nucleation line. Each metal via contacts one of the layer stack columns at its nucleation line. A low temperature dielectric liner extends along sidewalls of the layer stack columns. A non-conformal dielectric overlies the layer stack columns defining a plurality of air gaps between the layer stack columns.