A method includes forming a dummy gate over a substrate. A first gate spacer is formed on a side of the dummy gate. The dummy gate is replaced with a gate structure, such that that first gate spacer is on a side of the gate structure. The gate structure is etched back. After etching back the gate structure, a top portion of the first gate spacer is removed. A second gate spacer is formed over a remaining portion of the first gate spacer. After forming the second gate spacer, a dielectric cap is formed over the gate structure.