A paper sheet processing apparatus capable of preventing conveyance failure. A bill processing apparatus includes a bill insertion slot, a bill conveyance mechanism capable of conveying the bill inserted from the bill insertion slot, a bill reader reading the conveyed bill, a converter which converts an image read by the bill reader into data for each pixel as a unit of a predetermined size including color information having brightness, an authenticity judging mechanism determining an authenticity based on a density value for each pixel converted by the converter and a density value for each pixel of a reference bill, a bill determination processing part for determining a damage of the bill, before the bill reader completes reading, and a controller controlling the conveyance of the bill based on a determination result by the bill determination processing part.