A method of manufacturing a semiconductor device includes: forming an insulating film above a semiconductor substrate; forming a conductive film on the insulating film; forming a dielectric film on the conductive film; forming a plurality of upper electrodes at intervals on the dielectric film; forming a first protective insulating film on the upper electrodes and the dielectric film by a sputtering method; forming a second protective insulating film on the first protective insulating film by an atomic layer deposition method, thereby filling gaps of a grain boundary of the dielectric film with the second protective insulating film; and patterning the conductive film after the second protective insulating film is formed to provide a lower electrode.