A semiconductor device with a heterojunction structure having a substrate and a crystal layer which is grown over the substrate, in which a quantum dot buffer layer is interposed between the substrate and the crystal layer. In the semiconductor device, the interposition of the quantum dot buffer layer between the substrate and the crystal layer can effectively eliminate lattice mismatch between the substrate and the crystal layer. Therefore, a semiconductor device having excellent electro-optical characteristics can be obtained.