A nonvolatile semiconductor storage device has a semiconductor substrate, a gate electrode formed on a surface of the semiconductor substrate, and a first diffusion layer and a second diffusion layer formed in the surface of the semiconductor substrate on opposite sides of the gate electrode, a channel region being formed between the first and second diffusion layers. A first insulating layer, isolated pieces of material and a second insulating layer are formed in order in a multilayer structure on the surface of the semiconductor substrate on the channel region.