On a semiconductor substrate having an SOI region and a bulk silicon region formed on its upper surface, epitaxial layers are formed in source and drain regions of a MOSFET formed in the SOI region, and no epitaxial layer is formed in source and drain regions of a MOSFET formed in the bulk silicon region. By covering the end portions of the epitaxial layers with silicon nitride films, even when diffusion layers are formed by implanting ions from above the epitaxial layers, it is possible to prevent the impurity ions from being implanted down to a lower surface of a silicon layer.