A method of removing a hard mask is provided. Gate stacks are patterned on a substrate, where the gate stacks include a polysilicon layer and the hard mask deposited over the polysilicon layer. A dielectric layer is deposited on the substrate and on the patterned gate stacks. A first portion of the dielectric layer is planarized by chemical mechanical polishing (CMP) to remove a topography of the dielectric layer. The hard mask and a second portion of the dielectric layer are removed by the CMP.