A substrate including an NMOS transistor region and a PMOS transistor region is prepared. A silicon-germanium layer is formed on the PMOS transistor region. Nitrogen atoms are injected in an upper portion of the silicon-germanium layer. A first gate dielectric layer is formed on the NMOS transistor region and the PMOS transistor region. The nitrogen atoms are injected into the upper portion of the silicon-germanium layer before forming the first gate dielectric layer.