A method is provided for fabricating a semiconductor memory device. The method includes providing a substrate and forming a stacked layer on the substrate, where the stacked layer includes a tunneling dielectric layer and a floating gate layer sequentially formed on the substrate. The method also includes forming a plurality of stacked structures by etching the stacked layer and the substrate, where the spacing between the adjacent stacked structures forms a plurality of parallel first trenches. In addition, the method includes forming a plurality of second trenches and forming a plurality of third trenches. Moreover, the method includes forming a second dielectric layer on the floating gate layer and the side wall and bottom of the third trenches and forming a control gate layer on the second dielectric layer. Further, the method includes forming a plurality of fourth trenches and removing the sacrificial layer along the fourth trenches.