A vehicular vision system includes a monitoring system operable to determine a gaze direction of a driver of the vehicle. At least one vision sensor is disposed at the vehicle and has a field of view exterior of the equipped vehicle, and at least one non-vision sensor is disposed at the vehicle and has a field of sensing exterior of the vehicle. Image data captured by the interior viewing camera, image data captured by the exterior viewing camera and sensor data sensed by the non-vision sensor are provided to a control of the vehicle. A display system includes a video screen operable to display video images for viewing by the driver of the vehicle. The video screen also displays an augmented reality overlay. Responsive at least in part to processing of captured image data and/or sensed sensor data, the vision system determines the augmented reality overlay displayed.