To control a grain growth on laminated polysilicon films, a method of manufacturing a semiconductor device is provided. The method includes: forming a first polysilicon film (21) on a substrate (10); forming an interlayer oxide layer (22) on a surface of the first polysilicon film (21); forming a second polysilicon film (23) in contact with the interlayer oxide layer (22) above the first polysilicon film (21); and performing annealing at a temperature higher than a film formation temperature of the first and second polysilicon films in a gas atmosphere containing nitrogen, after formation of the second polysilicon film (23).