A semiconductor device is disclosed, which includes a semiconductor substrate including a device region and an isolation region having an isolation trench, a gate electrode formed on the device region through a gate insulating film, a first isolation insulating film formed in the isolation trench, the first isolation insulating film having a recess, a second isolation insulating film formed on the first isolation insulating film to be filled in the recess, the second isolation insulating film having an upper surface higher than the upper surface of the semiconductor substrate, and an impurity region formed in the semiconductor substrate under the first isolation insulating film, the impurity region having a conductivity type the same as a conductivity type of the semiconductor substrate, an impurity concentration higher than an impurity concentration of the semiconductor substrate, and a width of the impurity region smaller than a width of the isolation trench.