A method of forming an integrated circuit structure includes forming a first source/drain contact plug over and electrically coupling to a source/drain region of a transistor, forming a first dielectric hard mask overlapping a gate stack, recessing the first source/drain contact plug to form a first recess, forming a second dielectric hard mask in the first recess, recessing an inter-layer dielectric layer to form a second recess, and forming a third dielectric hard mask in the second recess. The third dielectric hard mask contacts both the first dielectric hard mask and the second dielectric hard mask.