A semiconductor device comprises a single crystal substrate, a nucleus formation buffer layer formed on the single crystal substrate, and a lamination layer including a plurality of Al.sub.1-x-y Ga.sub.x In.sub.y N (0.ltoreq.x.ltoreq.1, 0.ltoreq.y.ltoreq.1, x+y.ltoreq.1) layers laminated above the nucleus formation buffer layer. The nucleus formation buffer layer is formed of Al.sub.1-s-t Ga.sub.s In.sub.t N (0.ltoreq.s.ltoreq.1, 0.ltoreq.t.ltoreq.1, s+t.ltoreq.1) and is formed on a surface of the substrate such that the nucleus formation buffer layer has a number of pinholes for control of polarity and formation of nuclei. A method of fabricating a semiconductor device comprises the steps of: forming, above an Al.sub.1-x-y Ga.sub.x In.sub.y N (0.ltoreq.x.ltoreq.1, 0.ltoreq.y.ltoreq.1, x+y.ltoreq.1) semiconductor layer doped with a p-type dopant, a cap layer for preventing evaporation of a constituent element of the semiconductor layer, the cap layer being formed of one of AlN in which a p-type dopant is added and A1.sub.2 O.sub.3, subjecting the semiconductor layer to heat treatment, and removing at least a part of the cap layer.