The present invention relates to a polishing apparatus for removing surface roughness produced at a peripheral portion of a substrate, or for removing a film formed on a peripheral portion of a substrate. The polishing apparatus includes a housing for forming a polishing chamber therein, a rotational table for holding and rotating a substrate, a polishing tape supply mechanism for supplying a polishing tape into the polishing chamber and taking up the polishing tape which has been supplied to the polishing chamber, a polishing head for pressing the polishing tape against a bevel portion of the substrate, a liquid supply for supplying a liquid to a front surface and a rear surface of the substrate, and a regulation mechanism for making an internal pressure of the polishing chamber being set to be lower than an external pressure of the polishing chamber.