A computer implemented method for localization of an object, such as a license plate, in an input image includes generating a task-dependent representation of the input image based on relevance scores for the object to be localized. The relevance scores are output by a classifier for a plurality of locations in the input image, such as patches. The classifier is trained on patches extracted from training images and their respective relevance labels. One or more similar images are identified from a set of images, based on a comparison of the task-dependent representation of the input image and task-dependent representations of images in the set of images. A location of the object in the input image is identified based on object location annotations for the similar images.