In a memory cell region of a semiconductor device, a memory active region is defined by an element isolation insulating film. In the memory cell region, the position of the upper surface of the element isolation insulating film is set to be lower than the position of the main surface of a semiconductor substrate. A buried silicon nitride film and an etching stopper film are formed over the element isolation insulating film. The position of the upper surface of the etching stopper film is higher than that of the upper surface of the element isolation insulating film defining a peripheral active region.