An automatic emergency braking system for a vehicle includes a forward viewing camera and a control. At least in part responsive to processing of captured image data, the presence of another vehicle closing on the subject vehicle is determined, and a relative speed of the subject vehicle relative to the other vehicle is also determined. Responsive at least in part to a speed of the subject vehicle and the determined relative speed, the control controls the subject vehicle's brake system. Responsive to determination that the driver of the subject vehicle is impaired, the automatic emergency braking system does not allow the driver to override the control's control of the subject vehicle's brake system. Responsive to a determination that the driver of the subject vehicle is not impaired, the automatic emergency braking system allows the driver to override the control's control of the subject vehicle' brake system.