A semiconductor memory device includes a cell gate dielectric layer and a cell gate electrode disposed in a gate recess region crossing a cell active portion of a substrate, first and second doped regions disposed in the cell active portion at both sides of the gate recess region, respectively, at least one interlayer insulating layer covering the substrate, a data storage element electrically connected to the second doped region through a contact plug penetrating the at least one interlayer insulating layer, a mold layer covering the data storage element, and a bit line disposed in a cell groove formed in the mold layer. The bit line is in direct contact with a top surface of the data storage element.