Image modification styles learned from a limited set of modified images are described. A learned style system receives a selection of one or more modified images serving as a basis for a modification style. For each modified image, this system creates a modification memory, which includes a representation of the image content and modification parameters describing modification of this content to produce the modified image. These modification memories are packaged into style data, used to apply the modification style to input images. When applying a style, the system generates an image representation of an input image and determines measures of similarity between the input image's representation and representations of each modification memory in the style data. The system determines parameters for applying the modification style based, in part, on these similarity measures. The system modifies the input image according to the determined parameters to produce a styled image.