A semiconductor device and a method for forming the semiconductor device are provided. The method includes providing a substrate, and forming a first core layer on the substrate. The substrate includes a pull-up transistor region. The method also includes forming separately arranged second core layers on the first core layer, and forming a first sacrificial sidewall spacer on a sidewall of a second core layer. A gap is formed between adjacent first sacrificial sidewall spacers over the pull-up transistor region. In addition, the method includes removing the second core layers, and then etching the first core layer using the first sacrificial sidewall spacers as a mask until the substrate is exposed. The gap is transferred to a region between adjacent etched first core layers over the pull-up transistor region. Further, after etching the first core layer, the method includes forming a dielectric layer to fully fill the gap.