A semiconductor memory device comprises select transistors formed on side surfaces of plural silicon columns defined by a grid-like trenches on a surface of a silicon substrate, each select transistor having a source and a drain on the top surface and the bottom of the silicon column. A capacitor is formed on the top surface of the silicon column to form a DRAM cell. The source/drain layers on the bottom of a greater number of memory cells are commonly connected, or the source/drain layers on the bottom of adjacent memory cells are commonly connected, to be brought out to the surface of the silicon substrate by a connection line to be connected to a constant voltage or a bit line.