A semiconductor memory device for reducing parasitic bit line capacitance and a method of fabricating the same are provided. The semiconductor memory device includes a conductive pad formed on a semiconductor substrate and a first interlayer insulating layer having a first contact hole that exposes the conductive pad. The first interlayer insulating layer is formed on the conductive pad and the semiconductor substrate. Bit line stacks are formed on the first interlayer insulating layer. Bit line spacers are formed from a combination of materials having different dielectric constants on the sidewalls of the bit line stack to reduce the parasitic bit line capacitance. Preferably, the bit line spacers are stack layers including silicon nitride, silicon oxide, and silicon nitride. A second interlayer insulating layer having a second contact hole is formed on the bit line stack. A conductive plug fills the first and second contact holes. A storage electrode of a capacitor is formed on the conductive plug to be connected to the conductive pad.