An apparatus and method for detecting a region of interest in an image are disclosed. Image representations for a set of images that have been manually annotated with regions of interest are stored, along with positive and negative representations of each image which are similarly derived to the image representations except that they are based on features extracted from patches within the region of interest and outside it, respectively. For an original image for which a region of interest is desired, the stored information for K similar images is automatically retrieved and used to train a classifier. The trained classifier provides, for each patch of the original image, a probability of being in a region of interest, based extracted features of the patch (represented, for example, as a Fisher vector), which can be used to determine a region of interest in the original image.