A method of fabricating a semiconductor device includes the following steps. A substrate including at least a fin structure is provided, and a material layer is formed to cover the fin structure. Then, a first planarization process is performed on the material layer to form a first material layer, and an oxide layer is formed on the first material layer. Subsequently, the oxide layer is totally removed to expose the first material layer, and a second material layer is formed in-situ on the first material layer after totally removing the oxide layer.