In various examples, contrast values corresponding to pixels of one or more images generated using one or more sensors of a vehicle may be computed to detect and identify objects that trigger glare mitigating operations. Pixel luminance values are determined and used to compute a contrast value based on comparing the pixel luminance values to a reference luminance value that is based on a set of the pixels and the corresponding luminance values. A contrast threshold may be applied to the computed contrast values to identify glare in the image data to trigger glare mitigating operations so that the vehicle may modify the configuration of one or more illumination sources so as to reduce glare experienced by occupants and/or sensors of the vehicle.