A method of forming a semiconductor memory device includes stacking a plurality of alternating first insulating layers and first sacrificial layers on a substrate to form a first multilayer structure, forming a first hole through the first multilayer structure, forming a first semiconductor pattern in the first hole, stacking a plurality of alternating second insulating layers and second sacrificial layers on the first multilayer structure to form a second multilayer structure, forming a second hole through the second multilayer structure to be aligned with the first hole, forming a second semiconductor pattern in the second hole, forming a trench to expose sidewalls of the first and second insulating layers at a side of the first and second semiconductor patterns, removing at least some portions of the first and second sacrificial layers to form a plurality of recess regions, forming an information storage layer, and forming a conductive pattern.