The present invention provides a method for forming a semiconductor device having a metal gate. The method includes firstly, a substrate is provided, and a first semiconductor device and a second semiconductor device are formed on the substrate, having a first gate trench and a second trench respectively. Next, a bottom barrier layer is formed in the first gate trench and a second trench. Afterwards, a first pull back step is performed, to remove parts of the bottom barrier layer, and a first work function metal layer is then formed in the first gate trench. Next, a second pull back step is performed, to remove parts of the first work function metal layer, wherein the topmost portion of the first work function metal layer is lower than the openings of the first gate trench and the second gate trench.