A sheet processing apparatus includes: a conveying roller that conveys a sheet; a lateral registration skew correcting unit that corrects a position in a width direction orthogonal to a sheet conveying direction, of the sheet conveyed by the conveying roller; and a sheet processing portion that performs processing on the sheet conveyed by the conveying roller. A plurality of types of sheet processing portions is replaceable, and the sheet processing apparatus includes a puncher controller that changes an operation of the lateral registration skew correcting unit according to the type of the sheet processing portion.