A method of forming a semiconductor device includes the following steps. First of all, a material layer is formed on a substrate, and a sidewall image transferring process is performed to form plural first mask patterns on the material layer, with the first mask patterns parallel extended along a first direction. Next, a pattern splitting process is performed to remove a portion of the first mask patterns to form plural second openings, with the second openings parallel extended along a second direction, across the first mask patterns. Then, the material layer is patterned by using rest portions of the first mask patterns as a mask to form plural patterns arranged in an array.