A silicon semiconductor substrate, on which an epitaxial layer is to be formed, is set in a reaction vessel having a heating mechanism, and a gas containing TMG and AsH.sub.3 is introduced into the reaction vessel with the substrate heated to 450.degree. C., thus forming, on the substrate, a low-temperature growth layer of amorphous or polycrystalline GaAs as a semiconductor substance having a different lattice constant from that of the substrate. Then, with the TMG removed from the introduced gas, the temperature of the semiconductor substrate is increased to 750.degree. C., to cause coagulation of atoms of the low-temperature growth layer, with a thermal treatment also being performed at this high temperature, to cause growth of island-like single crystal cores. Further, a high temperature growth process is conducted in a material gas atmosphere containing TMG, whereby a GaAs film is epitaxially grown on the semiconductor substrate surface.