The invention provides a formation method of a semiconductor device. The method includes the following steps that: a semiconductor substrate is provided, and a gate structure is formed on the semiconductor substrate, wherein the gate structure includes a gate dielectric layer, a gate electrode layer on the gate dielectric layer and a hard mask layer on the gate electrode layer; an oxide layer is formed on the side wall surfaces of the gate electrode layer; the hard mask layer is etched, and a portion of the hard mask layer is removed, so that the shoulder of the gate electrode layer is exposed; a side wall material layer covering the bottom surface of the semiconductor substrate, the surface of the gate structure and the shoulder of the gate electrode layer, and a portion of the side wall material layer, which contacts with the shoulder, is at least partially formed by the material of the shoulder; and the side wall material layer on the bottom surface of the semiconductor substrate and the top surface of the gate structure is removed through adopting a dry etching process, and the side wall material layer at the surfaces of the side walls of the gate structure can form a side wall. With the formation method of the semiconductor device of the invention adopted, the defect of gate germanium-silicon residuals can be effectively eliminated.