One method disclosed herein includes forming at least one sacrificial sidewall spacer adjacent a sacrificial gate structure that is formed above a semiconducting substrate, removing at least a portion of the sacrificial gate structure to thereby define a gate cavity that is laterally defined by the sacrificial spacer, forming a replacement gate structure in the gate cavity, removing the sacrificial spacer to thereby define a spacer cavity adjacent the replacement gate structure, and forming a low-k spacer in the spacer cavity. A novel device disclosed herein includes a gate structure positioned above a semiconducting substrate, wherein the gate insulation layer has two upstanding portions that are substantially vertically oriented relative to an upper surface of the substrate. The device further includes a low-k sidewall spacer positioned adjacent each of the vertically oriented upstanding portions of the gate insulation layer.