One illustrative integrated circuit product disclosed herein includes a first final gate structure for a first transistor device, a second final gate structure for a second transistor device, the first and second transistors having a gate width direction and a gate length direction that is substantially normal to the gate width direction, and an insulating gate separation structure positioned between the first and second final gate structures, the insulating gate separation structure comprising an upper portion and a lower portion, the lower portion having a first lateral width in the gate width direction that is substantially uniform throughout a vertical height of the lower portion, the upper portion having a substantially uniform second lateral width in the gate width direction that is substantially uniform throughout a vertical height of the upper portion, wherein the second lateral width is less than the first lateral width.