In a vehicle remote operation system that remotely operates a vehicle V using a portable terminal 200, during the remote operation of the vehicle V by the portable terminal 200, when the cab of the vehicle V is not locked, a main control unit 112 of an on-board device 100 does not permit movement of the vehicle V. Therefore, the vehicle V can be remotely operated only when the cab of the vehicle is locked. As a result, it is possible to prevent a malicious person from getting in the vehicle V during the remote operation. In addition, even when a well-meaning person who gets in the cab unexpectedly unlocks and opens the door, the vehicle is stopped. Therefore, it is possible to ensure safety.