After a dummy control gate electrode and a memory gate electrode are formed and an interlayer insulating film is formed so as to cover the gate electrodes, the interlayer insulating film is polished to expose the dummy control gate electrode and the memory gate electrode. Thereafter, the dummy control gate electrode is removed by etching, and then a control gate electrode is formed in a trench which is a region from which the dummy control gate electrode has been removed. The dummy control gate electrode is made of a non-doped or n type silicon film, and the memory gate electrode is made of a p type silicon film. In the process of removing the dummy control gate electrode, the dummy control gate electrode is removed by performing etching under the condition that the memory gate electrode is less likely to be etched compared with the dummy control gate electrode, in the state where the dummy control gate electrode and the memory gate electrode are exposed.