A semiconductor memory device provided with a cell array section and a peripheral circuit section, the device includes: a back gate electrode; a stacked body provided on the back gate electrode; a plurality of semiconductor pillars extending in a stacking direction; connection members, each of the connection members connecting one of the semiconductor pillars to another one of the semiconductor pillars; a back-gate electrode contact applying a potential to the back gate electrode; a gate electrode provided in the peripheral circuit section; and a gate electrode contact applying a potential to the gate electrode, the back gate electrode and the gate electrode respectively including: a lower semiconductor layer; a conductive layer provided on the lower semiconductor layer; and an upper semiconductor layer provided on the conductive layer, the connection members being provided in or on the upper semiconductor layer, the back-gate electrode contact and the gate electrode contact being in contact with the conductive layer.