A sheet processing apparatus includes a first detector configured to detect a leading edge of a sheet conveyed in a conveying direction, a second detector provided on a downstream side of the first detector in the sheet conveying direction to detect the leading edge of the sheet conveyed, a punching portion, in the downstream side of the first detector in the sheet conveying direction, to move in a sheet width direction crossing the sheet conveying direction and perform a punching process for the sheet, an edge detector configured to move in the width direction together with the punching portion and detect the edge of the sheet conveyed in the width direction, and a controller, on the basis of at least either of information of a conveying speed of the sheet and a sheet length in the conveying direction, when the edge detector starts movement in the width direction after the first or second detector detects the leading edge of the sheet, to judge whether the edge detector can detect the edge of the sheet or not, as a result of the judgment, selecting the detector positioned on the most downstream side in the conveying direction among the first and second detectors which can be used, and when the selected first or second detector detects the leading edge of the sheet conveyed, permitting the edge detector to start movement to detect the edge of the sheet in the width direction.