A semiconductor structure and methods for forming the same are provided. The semiconductor structure includes a first MOS device of a first conductivity type and a second MOS device of a second conductivity type opposite the first conductivity type. The first MOS device includes a first gate dielectric on a semiconductor substrate; a first metal-containing gate electrode layer over the first gate dielectric; and a silicide layer over the first metal-containing gate electrode layer. The second MOS device includes a second gate dielectric on the semiconductor substrate; a second metal-containing gate electrode layer over the second gate dielectric; and a contact etch stop layer having a portion over the second metal-containing gate electrode layer, wherein a region between the portion of the contact etch stop layer and the second metal-containing gate electrode layer is substantially free from silicon.