A method and apparatus of forming a nonvolatile semiconductor device including forming a first gate insulating film on a main surface of a first semiconductor region, forming a first gate electrode on the first gate insulating film, forming a second gate insulating film, forming a second gate electrode over a first side surface of the first gate electrode, selectively removing the second gate insulating film, etching the second gate insulating film kept between the second gate electrode and a main surface of the first semiconductor region in order to form an etched charge storage layer, introducing first impurities in the first semiconductor region in a self-aligned manner to the second gate electrode in order to form a second semiconductor region, annealing the semiconductor substrate to extend the second semiconductor region to an area under the second gate electrode.