A paper sheet conveyance device includes a plurality of conveyance members and conveys a paper sheet while performing temporary stop of conveying and then restart of conveying. Two paper sheets sequential in a conveyance direction stop together in a state of the temporary stop, driving a first transfer member to which a preceding first paper sheet that is one of the two paper sheets is opposed and a second conveyance member to which a following second paper sheet that is the other paper sheet is opposed is stopped and restarted synchronously for the temporary stop and the restart. A stop position information acquisition unit acquires information on a position on the conveyance path at which the second paper sheet stops in the state of the temporary stop, and a timing for stopping the conveying after the conveying is restarted is controlled according to the acquired information.