One illustrative method disclosed herein includes removing the sidewall spacers and a gate cap layer so as to thereby expose an upper surface and sidewalls of a sacrificial gate structure, forming an etch stop layer above source/drain regions of a device and on the sidewalls and upper surface of the sacrificial gate structure, forming a first layer of insulating material above the etch stop layer, removing the sacrificial gate structure so as to define a replacement gate cavity that is laterally defined by portions of the etch stop layer, forming a replacement gate structure in the replacement gate cavity, and forming a second gate cap layer above the replacement gate structure.