A method of manufacturing a semiconductor device includes: preparing a stacked body in which a first layer, a second layer, a third layer, and a fourth layer are stacked in this order on a semiconductor substrate in a first direction, the stacked body including a first region and a second region different from the first region; etching the fourth layer in the first region and the second region to expose the third layer by irradiating the first region and the second region with an ion beam, and etching the third layer and the second layer in the second region to expose the first layer by irradiating the second regions with an ion beam in a state where the third layer is exposed in the first region.