In one embodiment, a method of manufacturing a semiconductor device includes forming a stacked body that alternately includes a plurality of first films and a plurality of second films on a substrate. The method further includes performing a first process of forming N2 holes having N kinds of depths in the stacked body where N is an integer of three or more. The method further includes performing a second process of processing the N2 holes so as to have N2 kinds of depths after performing the first process.