The present invention provides a sheet processing apparatus, such as a finisher or sorter, for implementing predetermined processing such as alignment and stapling on an inserted sheet material (i.e. any sheet-form recording medium such as recording paper, transfer paper, or an OHP sheet), and an image forming apparatus comprising the sheet processing apparatus. The sheet processing apparatus comprises a rear end fence horizontal portion that contacts the end portion of the sheet member on the upstream side of a sheet member conveyance direction during alignment of the sheet member, a rear end fence vertical portion for supporting the other parts of the sheet member, and a staple unit for stapling the sheet member. The stapling direction in which a staple is punched by a stapler of the staple unit is set to be parallel to the rear end fence horizontal portion that contacts the rear end portion of the sheet member.