A method of manufacturing a semiconductor memory device of an embodiment includes: after forming a first interconnection layer and a memory cell layer above a semiconductor substrate, forming first lines by forming first grooves extending in first direction; forming a thin film on the side walls of the first grooves; forming a stack structure by filling an interlayer insulating film in the first grooves; forming a second interconnection layer above the stack structure; forming second lines by forming second grooves extending in second direction; removing the thin film exposed at bottom of the second grooves; and forming columnar memory cells by removing the memory cell layer exposed at bottom of the second grooves. The thin film has higher etching rate than the interlayer insulating film, and is removed prior to portions of the memory cell layer adjoining the thin film.