A method for identifying digital images having matching backgrounds from a collection of digital images, comprising using a processor to perform the steps of: determining a set of one or more feature values for each digital image in the collection of digital images, wherein the set of feature values includes an edge compactness feature value that is an indication of the number of objects in the digital image that are useful for scene matching; determining a subset of the collection of digital images that are good candidates for scene matching by applying a classifier responsive to the determined feature values; applying a scene matching algorithm to the subset of the collection of digital images to identify groups of digital images having matching backgrounds; and storing an indication of the identified groups of digital images having matching backgrounds in a processor-accessible memory.