According to an embodiment, a semiconductor memory device comprises control gate electrodes and a semiconductor layer. The control gate electrodes are stacked above a substrate. The semiconductor layer has as its longitudinal direction a direction perpendicular to the substrate. The semiconductor memory device further comprises first and second control gate electrodes and third and fourth control gate electrodes stacked sequentially above the substrate and first through fourth via contacts connected to these first through fourth control gate electrodes. The third and fourth control gate electrodes face the first and second control gate electrodes. Positions of the first and second via contacts are far from each other. Positions of the third and fourth via contacts are close to each other.