A method of fabricating a metal gate structure is provided. The method includes providing a semiconductor substrate with a planarized polysilicon material; patterned the planarized polysilicon material to form at least a first gate and a second gate, wherein the first gate is located on the active region and the second gate at least partially overlaps with the isolation region; forming an inter-layer dielectric material covering the gates; planarizing the inter-layer dielectric material until exposing the gates and forming an inter layer-dielectric layer; performing an etching process to remove the gates to form a first recess and a second recess within the inter-layer dielectric layer; forming a gate dielectric material on a surface of each of the recesses; forming at least a metal material within the recesses; and performing a planarization process.