A semiconductor device includes a substrate, a first isolation structure, a second isolation structure and a dummy pattern. The substrate includes a first part surrounding a second part at a top view. The first isolation structure is disposed between the first part and the second part, to isolate the first part from the second part. The second isolation structure is disposed at at least one corner of the first part. The dummy pattern is disposed on the second isolation structure. The present invention also provides a method of forming said semiconductor device.