First through holes of a relatively small diameter and second through holes of a relatively great diameter are formed in proper shapes by separate processes, respectively, in a first layer insulating film. The second through holes are tapered toward a layer underlying the first layer insulating film. First, the first through holes are formed in the first layer insulating film, the first through holes are filled up with plug electrodes, and the second through holes are formed in the first layer insulating film. When filling up the first and the second through holes formed in the first layer insulating film with plug electrodes, a first conductive film deposited over the first layer insulating film is etched back to fill up the first through holes with the plug electrodes, and then etch back residues remaining on the side walls of the second through holes are removed.