A semiconductor device having a transistor of gate all around (GAA) type and a method of fabricating the same are disclosed. A SOI substrate composed of a SOI layer, a buried oxide layer and a lower substrate is prepared. The SOI layer has at least one unit dual layer of a silicon germanium layer and a silicon layer. The SOI layer is patterned to form an active layer pattern to a certain direction. An insulation layer is formed to cover the active layer pattern. An etch stop layer is stacked on the active layer pattern covered with the insulation layer. The etch stop layer is patterned and removed at a gate region crossing the active layer pattern at the channel region. The insulation layer is removed at the gate region. The silicon germanium layer is isotropically etched and selectively removed to form a cavity at the channel region of the active layer pattern. In the state that the silicon germanium layer is selectively removed, a gate insulation layer is formed to cover the exposed surface of the active layer pattern. A gate conductivity layer is stacked on the substrate by a chemical vapor deposition (CVD) to fill the gate region including the cavity. The middle part of the channel region of the active layer pattern can be patterned to be divided by multiple patterns that are formed in a line.