An image of an object from a known object class is synthesized by first obtaining reflectance fields for various training objects from the object class. A reflectance field model is defined for the object class using a combination of the reflectance fields of the training objects. The parameters of the reflectance field model are optimized to estimate a particular reflectance field of a particular object from the object class given one or more input images of the particular object. The particular reflectance field is fitted to the particular object, and then the new image of the particular object is synthesized by changing the illumination parameters of the particular fitted reflectance field model after the fitting.