A structure and a formation method of a semiconductor device are provided. The semiconductor device includes a semiconductor substrate and a fin structure over the semiconductor substrate. The semiconductor device also includes a gate stack covering a portion of the fin structure and an epitaxially grown source/drain structure over the fin structure and adjacent to the gate stack. The semiconductor device further includes a semiconductor protection layer over the epitaxially grown source/drain structure. The semiconductor protection layer has an atomic concentration of germanium greater than that of the epitaxially grown source/drain structure.