The present disclosure provides a method of fabricating a semiconductor device. The method includes forming a first gate structure and a second gate structure over a substrate. The first and second gate structures each include a high-k dielectric layer located over the substrate, a capping layer located over the high-k dielectric layer, an N-type work function metal layer located over the capping layer, and a polysilicon layer located over the N-type work function metal layer. The method includes forming an inter-layer dielectric (ILD) layer over the substrate, the first gate structure, and the second gate structure. The method includes polishing the ILD layer until a surface of the ILD layer is substantially co-planar with surfaces of the first gate structure and the second gate structure. The method includes replacing portions of the second gate structure with a metal gate. A silicidation process is then performed to the semiconductor device.