A semiconductor memory device, and a method of manufacturing the semiconductor memory device, includes a gate stack including interlayer insulating layers and word lines alternately stacked in a first direction, channel pillars passing through the gate stack and tapering toward the first direction, source select lines surrounding the channel pillars and extending to overlap the gate stack, and a source isolation insulating layer overlapping the gate stack between the source select lines and tapering toward a direction opposite to the first direction.