The polishing apparatus has a polishing unit capable of polishing a peripheral portion of the substrate to form a right-angled cross section. The polishing apparatus includes: a substrate holder that holds and rotates the substrate; guide rollers that support a polishing tape; and a polishing head having a pressing member that presses an edge of the polishing tape against the peripheral portion of the substrate from above. The guide rollers are arranged such that the polishing tape extends parallel to a tangential direction of the substrate and a polishing surface of the polishing tape is parallel to a surface of the substrate. The substrate holder includes: a holding stage that holds the substrate; and a supporting stage that supports a lower surface of the peripheral portion of the substrate in its entirety. The supporting stage rotates in unison with the holding stage.