A vehicular driver monitoring system includes a driver status information acquisition system and an in-vehicle control system of the vehicle. The driver status information acquisition system receives data from a plurality of sensors in the vehicle and determines the driver status responsive to processing of the received data. While the driver is not driving the vehicle and the in-vehicle control system is autonomously driving the vehicle, and responsive to determination that the driver of the vehicle should take over driving the vehicle from the in-vehicle control system, the in-vehicle control system alerts the driver to indicate that the driver should take over driving the vehicle. Responsive to the status of the driver while the driver is not driving the vehicle being indicative of the driver not being able to take over driving the vehicle, the in-vehicle control system continues driving the vehicle and initiates an emergency action.