This disclosure relates to a system and method for determining vehicle operator preparedness for vehicles that support both autonomous operation and manual operation. The system includes sensors configured to generate output signals conveying information related to vehicles and their operation. During autonomous vehicle operation, the system gauges the level of responsiveness of an individual vehicle operator through challenges and corresponding responses. Based on the level of responsiveness, a preparedness metric is determined for each vehicle operator individually.