Each gate structure formed on the substrate includes a gate dielectric, a gate conductor, a first etch stop layer, and a gate cap dielectric. A second etch stop layer is formed over the gate structures, gate spacers, and source and drain regions. A first contact-level dielectric layer and a second contact-level dielectric layer are formed over the second etch stop layer. Gate contact via holes extending at least to the top surface of the gate cap dielectrics are formed. Source/drain contact via holes extending to the interface between the first and second contact-level dielectric layers are subsequently formed. The various contact via holes are vertically extended by simultaneously etching exposed gate cap dielectrics and exposed portions of the first contact-level dielectric layer, then by simultaneously etching the first and second etch stop layers. Source/drain contact vias self-aligned to the outer surfaces gate spacers are thereby formed.