A semiconductor memory device having memory cells formed adjacent to each other comprises a P type semiconductor substrate having adjacent two trenches, a P.sup.+ impurity region formed in the side portions and the bottom portions of the trenches, n type first polysilicon layers serving as common electrodes formed in the upper portion of the P.sup.+ impurity region through an insulating film, second polysilicon layers formed inside and in the upper portion of the trenches formed of the first polysilicon layers through an insulating film, and a third polysilicon layer formed on the second polysilicon layers, only the third polysilicon layer constituting a connecting electrode between the adjacent memory cells.