A sheet processing apparatus capable of high-speed and accurate detection of a lateral displacement and a skew of a sheet. A motor laterally shifts the sensors during sheet conveyance by a conveying motor, to determine first and second positions of the sheet lateral edge detected respectively by first and second sensors. A finisher controller determines a third position of the lateral edge of the sheet closer to a sheet trailing edge, based on the first and second positions and an amount of conveyance of the sheet from when the first position is detected to when the second position is detected. A lateral displacement of the sheet is corrected by laterally shifting the sheet according to the third position of the sheet lateral edge.