A wide band gap semiconductor device having a JFET, a MESFET, or a MOSFET mainly includes a semiconductor substrate, a first conductivity type semiconductor layer, and a first conductivity type channel layer. The semiconductor layer is formed on a main surface of the substrate. A recess is formed in the semiconductor layer in such a manner that the semiconductor layer is divided into a source region and a drain region. The recess has a bottom defined by the main surface of the substrate and a side wall defined by the semiconductor layer. The channel layer has an impurity concentration lower than an impurity concentration of the semiconductor layer. The channel layer is formed on the bottom and the side wall of the recess by epitaxial growth.