A method of fabricating a semiconductor device comprises: forming an etch stop layer to cover sidewall and top surfaces of first and second dummy gate patterns on a substrate; and forming an interlayer insulating layer on the substrate and the etch stop layer. The interlayer insulating layer is planarized to expose the etch stop layer on the first and second dummy gate patterns, and the etch stop layer is etched to expose the top surfaces and upper sidewall surfaces of the first and second dummy gate patterns, thereby forming a groove between the interlayer insulating layer and the first and second dummy gate patterns. The dummy gate patterns are removed, and gate electrodes are formed in their places.