A semiconductor device with a nonvolatile memory is provided which has improved characteristics. The semiconductor device includes a control gate electrode, a memory gate electrode disposed adjacent to the control gate electrode, a first insulating film, and a second insulating film including therein a charge storing portion. Among these components, the memory gate electrode is formed of a silicon film including a first silicon region positioned over the second insulating film, and a second silicon region positioned above the first silicon region. The second silicon region contains p-type impurities, and the concentration of p-type impurities of the first silicon region is lower than that of the p-type impurities of the second silicon region.