A method of manufacturing a semiconductor device includes the following steps. A first patterned photoresist layer is formed on a substrate. A second patterned photoresist layer is formed on the substrate after the first patterned photoresist layer is formed, wherein the first patterned photoresist layer and the second patterned photoresist layer are arranged alternatively. A liner is formed to cover sidewalls of the first patterned photoresist layer and the second patterned photoresist layer. The present invention also provides a semiconductor device, including a plurality of pillars being disposed on a layer, wherein the layer includes first recesses and second recesses, wherein the depths of the first recesses are less than the depths of the second recesses.