Provided is a semiconductor device including a substrate, an insulating layer, a conductive layer and at least one spacer. The substrate has at least two shallow trenches therein. The conductive layer is disposed on the substrate between the shallow trenches. The insulating layer is disposed between the substrate and the conductive layer. The at least one spacer is disposed on one sidewall of the conductive layer and fills up each shallow trench. A method of forming a semiconductor device is further provided.