A semiconductor structure includes a plurality of semiconductor fins located on a semiconductor substrate, in which each of the semiconductor fins comprises a sequential stack of a buffered layer including a III-V semiconductor material and a channel layer including a III-V semiconductor material. The semiconductor structure further includes a gap filler material surrounding the semiconductor fins and including a plurality of trenches therein. The released portions of the channel layers of the semiconductor fins located in the trenches constitute nanowire channels of the semiconductor structure, and opposing end portions of the channel layers of the semiconductor fins located outside of the trenches constitute a source region and a drain region of the semiconductor structure, respectively. In addition, the semiconductor structure further includes a plurality of gates structures located within the trenches that surround the nanowire channels in a gate all around configuration.