A system and method for automatically removing polarized light in an environment having polarized light and unpolarized light. Light is processed by a polarizer and measured by a sensor. Multiple measurements of a light characteristic are taken, each measurement corresponding to a polarity setting of the polarizer. The polarizer is automatically adjusted to enable the multiple measurements. The measurements may be of average light intensity, contrast, saturation, or another characteristic. Based on the multiple measurements and the corresponding polarity settings, an optimal polarity setting is determined, such that the amount of polarized light is minimized. A curve fitting calculation may be used to make the determination. The polarizer is adjusted to the determined polarity setting.