A system and method that generates a candidate list of indexed images that potentially match an object in a query image is disclosed. The method includes receiving a query image including an object, receiving a plurality of indexed images that match the object, computing a region of interest for the object, computing an overlap between a first region of interest corresponding to a first indexed image and a second region of interest corresponding to a second indexed image, determining that the overlap between the first region of interest and the second region of interest satisfies a threshold and including the first indexed image and the second indexed image in a candidate list of indexed images.