A prime mover control device of a construction machine includes a hydraulic pump (24) driven by a prime mover (10), an actuator (5) driven with pressure oil discharged from the hydraulic pump (24), and a control valve (25) that controls a flow of the pressure oil from the hydraulic pump (24) to the actuator (5) in response to an operation of a first operating member (22a). The prime mover control device includes a first set means (41) that sets a first set rotation speed Nt of the prime mover (10) according to the operation of the first operating member (22a), a second set means (43) that sets a second set rotation speed Nx of the prime mover (10) according to an operation of a second operating member (34), a selection member (35) that selects one of a first mode and a second mode, and a rotation speed control means (30, 13) that controls a prime mover rotation speed to match with a maximum value of the first set rotation speed Nt and the second set rotation speed Nx when the selection member (35) selects the first mode, and controls the prime mover rotation speed to match with the second set rotation speed Nx when the selection member (35) selects the second mode.