A semiconductor memory device according to an embodiment includes first and second conductive layers and first and second pillar. The first pillar penetrates the first conductive layers, and includes one part of a first semiconductor layer. The second pillar penetrates the second conductive layer and is provided on the first pillar. The second pillar includes another part of the first semiconductor layer. An area of the second pillar is smaller than an area of the first pillar. The first semiconductor layer includes a first portion facing an uppermost one of the first conductive layers and a second portion facing the second conductive layer. The first semiconductor layer is continuous at least from the first portion to the second portion.