This disclosure relates to a system and method for transitioning vehicle control between autonomous operation and manual operation. The system includes sensors configured to generate output signals conveying information related to the vehicle and its operation. During autonomous vehicle operation, the system gauges the level of responsiveness of a vehicle operator through challenges and corresponding responses. The system determines when to present a challenge to the vehicle operator based on internal and external factors. If necessary, the system will transition from an autonomous operation mode to a manual operation mode.