A semiconductor memory device has a semiconductor substrate of one conductivity type in which a plurality of memory cells are formed, each of the plurality of memory cells including at least one capacitor and having a trench which is formed from one major surface of the semiconductor substrate so as to surround at least one memory cell, wherein a first insulating film having element isolation properties is formed on a bottom and most areas of side wall surfaces of the trench, a first conductive film serving as one electrode of the capacitor is formed on the side wall of the first insulating film and an exposed portion of the semiconductor substrate which is not covered with the first insulating film, a second insulating film is formed on the first conductive film, and a second conductive film serving as the other electrode of the capacitor is formed on the second insulating film.