The present disclosure provides a semiconductor device and a method for forming a gate structure thereof. The method includes: preparing a semiconductor substrate, and forming an active region on the semiconductor substrate; forming a dummy gate stack, a gate sidewall spacer, N-type and/or P-type source/drain regions, and an interlayer dielectric layer on the active region sequentially; removing the dummy gate stack to form a gate opening, and forming an interface oxide layer and a ferroelectric gate dielectric layer sequentially at the gate opening; forming a stress sacrificial layer on the ferroelectric gate dielectric layer, and performing an annealing process; during the annealing process, the clamping effect of the stress sacrificial layer induces the ferroelectric gate dielectric layer converted to form a ferroelectric-phase gate dielectric layer; removing the stress sacrificial layer; and forming a metal gate on the ferroelectric-phase gate dielectric layer.