A polishing apparatus which can efficiently polish an entirety of a back surface of a substrate, with the back surface facing downward, is disclosed. The polishing apparatus includes: a substrate holder configured to rotate the substrate; a polishing head configured to polish the back surface of the substrate; a tape advancing device; and a translational rotating mechanism configured to cause the polishing head to make a translational rotating motion. The substrate holder includes a plurality of rollers which are rotatable about their own axes. The plurality of rollers have substrate-holding surfaces capable of contacting a periphery of the substrate. The polishing head is disposed below the substrate-holding surfaces. The polishing head includes a polishing blade configured to press the polishing tape against the back surface of the substrate, and a pressing mechanism configured to push the polishing blade upward.