The first impurity species having a low diffusion rate is heavily doped in a predetermined region of a semiconductor substrate in contact with portions corresponding to the edges of a floating gate, and the second impurity species having a low diffusion rate is lightly doped in the predetermined region from a position separated from the portions corresponding to the edges of the floating gate by a predetermined distance. Annealing is performed such that the second impurity species is diffused below the floating gate more inward than the first impurity species, and part of a diffusion region formed by the first impurity species serves as a tunnel region which overlaps the floating gate. With this structure, a short channel effect can be prevented, and an inter-band current can be suppressed.