The upper end of a gate electrode is situated below the surface of a semiconductor substrate. An insulating layer is formed over the gate electrode and over the semiconductor substrate situated at the periphery thereof. The insulating layer has a first insulating film and a low oxygen permeable insulating film. The first insulating film is, for example, an NSG film and the low oxygen permeable insulating film is, for example, an SiN film. Further, a second insulating film is formed over the low oxygen permeable insulating film. The second insulating film is, for example, a BPSG film. The TDDB resistance of a vertical MOS transistor is improved by processing with an oxidative atmosphere after forming the insulating layer. Further since the insulating layer has the low oxygen permeable insulating film, fluctuation of the threshold voltage of the vertical MOS transistor can be suppressed.