In a semiconductor device (MISFET) having a gate electrode formed over a nitride semiconductor layer with a gate insulating film interposed therebetween, the gate insulating film includes a first gate insulating film (oxide film of first metal) formed on the nitride semiconductor layer and a second gate insulating film (oxide film of second metal). The second metal (for example, Hf) has electronegativity lower than that of the first metal (for example, Al). Since the electronegativity of the second metal is lower than that of the first metal, negative charge is introduced into the oxide film of the first metal due to interfacial polarization, so that the flat-band voltage can be shifted in a positive direction. Accordingly, the threshold voltage which has become negative due to the heat treatment of the oxide film of the first metal can be shifted in the positive direction.