A semiconductor device including a substrate, a plurality of isolation structures, at least a gate structure, a plurality of dummy gate structures and a plurality of epitaxial structures is provided. The substrate has an active area defined by the isolation structures disposed within the substrate. That is, the active area is defined between the isolation structures. The gate structure is disposed on the substrate and located within the active area. The dummy gate structures are disposed on the substrate and located out of the active area. The edge of each dummy gate structure is separated from the boundary of the active area with a distance smaller than 135 angstroms. The epitaxial structures are disposed within the active area and in a portion of the substrate on two sides of the gate structure. The invention also provided a method for fabricating semiconductor device.