A semiconductor device structure is provided. The semiconductor device structure includes a transistor which includes a source/drain feature adjoining an active region, and a gate stack over the active region. The semiconductor device structure further includes a capacitor above the transistor, the capacitor including a bottom electrode layer on the gate stack and a ferroelectric layer on the bottom electrode layer. The ferroelectric layer is made of a Hf-based dielectric material. The semiconductor device structure further includes gate spacer layers surrounding the gate stack, the bottom electrode layer and the ferroelectric layer.