A first conductive type layer having a band gap energy smaller than that of an under growth layer formed on a substrate is formed by selective growth from an opening portion formed in the under growth layer, and an active layer and a second conductive type layer are stacked on the first conductive type layer, to form a stacked structure. When such a stacked structure for forming a semiconductor device is irradiated with laser beams having an energy value between the band gap energies of the under growth layer and the first conductive type layer, abrasion occurs at a first conductive type layer side interface between the under growth layer and the first conductive type layer, so that the stacked structure is peeled from the substrate and the under growth layer and simultaneously isolated from another stacked structure for forming another semiconductor device. Since the first conductive layer has good crystallinity and is suitable for formation of an electrode thereon, an electrode can be efficiently formed on the back surface of the first conductive type layer of the peeled stacked structure.