A vision system of a vehicle includes a camera disposed at a vehicle and having a field of view forward of the vehicle. An image processing system is operable to process image data captured by the camera to determine the presence of a leading vehicle ahead of the equipped vehicle and to determine an illumination level of a taillight of the leading vehicle. Responsive to a determination of a relative speed of the leading vehicle relative to the equipped vehicle and a relative acceleration of the leading vehicle relative to the equipped vehicle, the vision system determines a braking level of the equipped vehicle to mitigate collision with the leading vehicle. Responsive to the determination of the illumination level of the taillight of the determined leading vehicle, the vision system is operable to apply a weighting factor to adjust the determined braking level for the determined traffic condition.