According to an embodiment, a semiconductor memory device includes a plurality of control gate electrodes, a semiconductor layer, and a charge accumulation layer. The plurality of control gate electrodes are provided as a stack above a substrate. The semiconductor layer has as its longitudinal direction a direction perpendicular to the substrate, and faces the plurality of control gate electrodes. The charge accumulation layer is positioned between the control gate electrode and the semiconductor layer. A lower end of the charge accumulation layer is positioned more upwardly than a lower end of a lowermost layer-positioned one of the control gate electrodes.