A semiconductor memory device includes a first electrode layer; a second electrode layer provided above the first electrode layer; a first insulating oxide layer provided between the first and second electrode layers; a semiconductor layer extending through the first electrode layer, the first insulating oxide layer and the second electrode layer that are stacked in the first direction; and a second insulating oxide layer extending in the first direction between the semiconductor layer and the first insulating oxide layer, the second insulating oxide layer being in contact with the first insulating oxide layer. At least one of the first insulating oxide layer and the second insulating oxide layer includes nitrogen atoms. The nitrogen atoms are distributed around an interface between the first insulating oxide layer and the second insulating oxide layer, or distributed in the vicinity of the interface.