One illustrative method disclosed herein includes, among other things, forming a fin that is positioned above and vertically spaced apart from an upper surface of a semiconductor substrate, the fin having an upper surface, a lower surface and first and second side surfaces, wherein an axis of the fin in a height direction of the fin is oriented substantially parallel to the upper surface of the substrate, and wherein a first side surface of the fin contacts a first insulating material, forming a gate structure around the upper surface, the second side surface and the lower surface of the fin, and forming a gate contact structure that is conductively coupled to the gate structure.