A method, data processing system, and computer program product for managing a headlight on a vehicle are presented. An image of an area in front of the vehicle is received. A first set of features is identified in the received image. The first set of features in the received image is compared with a number of sets of features from a plurality of previous images. Each image in the plurality of previous images is associated with a headlight setting. A second set of features from a previous image in the plurality of previous images matching the first set of features in the received image is identified. A determination is made whether to change a setting for the headlight on the vehicle based on the headlight setting associated with the previous image.