Techniques are described for white balancing an input image by determining a color transformation for the input image based on color transformations that have been computed for training images whose color characteristics are similar to those of the input image. Techniques are also described for generating a training dataset comprising color information for a plurality of training images and color transformation information for the plurality of training images. The color information in the training dataset is searched to identify a subset of training images that are most similar in color to the input image. The color transformation for the input image is then computed by combining color transformation information for the identified training images. The contribution of the color transformation information for any given training image to the combination can be weighted based on the degree of color similarity between the input image and the training image.