A nonvolatile semiconductor memory device includes: a substrate; a memory multilayer body with a plurality of insulating films and electrode films alternately stacked therein, the memory multilayer body being provided on a memory array region of the substrate; a semiconductor pillar buried in the memory multilayer body and extending in stacking direction of the insulating films and the electrode films; a charge storage film provided between one of the electrode films and the semiconductor pillar; a dummy multilayer body with a plurality of the insulating films and the electrode films alternately stacked therein and a dummy hole formed therein, the dummy multilayer body being provided on a peripheral circuit region of the substrate; an insulating member buried in the dummy hole; and a contact buried in the insulating member and extending in the stacking direction.