A method of fabricating a semiconductor device according to one embodiment includes: removing a native oxide film and adhering silicon nitrides on an area of a Si based substrate in hydrogen gas atmosphere under a condition in which a pressure is a first pressure and a temperature is a first temperature, a silicon nitride-containing member being formed on the Si based substrate, the area being a area not covered by the member; lowering the temperature to a second temperature from the first temperature while maintaining the pressure at the first pressure in hydrogen gas atmosphere; lowering the pressure to a second pressure from the first pressure while maintaining the temperature at the second temperature in hydrogen gas atmosphere; and epitaxially growing a crystal on the area of the Si based substrate in a precursor gas atmosphere after the pressure is lowered to the second pressure, the crystal including at least one of Si and Ge, the precursor gas atmosphere including at least one of hydrogen, Si and Ge.