The invention relates to a semiconductor memory device which comprises a base plate of a first impurity type, a first trap region of a second impurity type different from the first impurity type in the base plate, a second trap region of the first impurity type in the base plate, a first patterned dielectric layer extended above the first trap region and the second trap region on the base plate, a first patterned grid structure on the first patterned dielectric layer, a second patterned dielectric layer on the first patterned grid structure, and a second patterned grid structure on the second patterned dielectric layer. The first patterned grid structure can comprise a first section extending towards a first direction and a second section extending towards a second direction perpendicular to the first section, and the first section and the second section intersect with each other to form an intersecting pattern. The second patterned grid structure can comprise at least one of a first section extending above the first patterned grid structure towards a first direction and a second section extending above the first patterned grid structure towards a second direction.