In a method of manufacturing a semiconductor device comprising melting an amorphous or polycrystalline first semiconductor layer formed on the surface of a first dielectric layer by irradiating energy rays thereon, and converting the same into single crystals by the subsequent lowering of the temperature and forming a second dielectric layer and a second semiconductor layer on the first semiconductor layer. Energy rays are irradiated under the condition capable of melting the first semiconductor layer through the second semiconductor layer and the second dielectric layer and, after the completion of the conversion into single crystals, the second semiconductor layer and the second dielectric layer are eliminated through etching.