Methods, systems and apparatus for identifying modified images based on visual dissimilarity to a first image. In an aspect, a method includes determining, for each of a first image and a second image, a respective set of local image feature descriptions; determining one or more unmatched regions of the images that include unmatched image features and that correspond to one or more same respective regions in both the first image and the second image; determining, for each of the one or more unmatched regions of the images, a modification measure based on the image data corresponding to the unmatched region in the first image and the image data corresponding to the unmatched region in the second image; and determining that the second image is a modification of the first image when one of the modification measures meets a modification measure threshold.