A semiconductor memory device includes pillars extending upright on a substrate in a direction perpendicular to the substrate, a stack disposed on the substrate and constituted by a first interlayer insulating layer, a first conductive layer, a second interlayer insulating layer, and a second conductive layer, a variable resistance layer interposed between the pillars and the first conductive layer, and an insulating layer interposed between the first pillars and the second conductive layer.