A semiconductor device includes a semiconductor substrate, a transistor, a conductive contact and a capacitor. The transistor is formed on the semiconductor substrate, and the transistor includes a gate, a source and a drain. The conductive contact is formed on and in contact with at least one of the source and the drain. The capacitor includes a first electrode and a second electrode spaced apart from first electrode. At least one of the first and second electrodes extends on substantially the same level as the conductive contact or the gate. A method of forming the semiconductor device is provided as well.