A semiconductor device includes a gate electrode disposed on a semiconductor substrate and source/drain regions disposed at both sides of the gate electrode, the source/drain regions being formed by implanting impurities. The source/drain regions include an epitaxial layer formed by epitaxially growing a semiconductor material having a different lattice constant from that of the semiconductor substrate in a recessed position at a side of the gate electrode, and a diffusion layer disposed in a surface layer of the semiconductor substrate.