The present invention provides a polishing apparatus for polishing a periphery of a substrate. This polishing apparatus includes a rotary holding mechanism configured to hold the substrate horizontally and rotate the substrate, plural polishing head assemblies provided around the substrate, plural tape supplying and recovering mechanisms configured to supply polishing tapes to the plural polishing head assemblies and recover the polishing tapes from the plural polishing head assemblies, and plural moving mechanisms configured to move the plural polishing head assemblies in radial directions of the substrate held by the rotary holding mechanism. The tape supplying and recovering mechanisms are located outwardly of the plural polishing head assemblies in the radial directions of the substrate, and the tape supplying and recovering mechanisms are fixed in position.