Described herein is a framework for multi-view matching of regions of interest in images. According to one aspect, a processor receives first and second digitized images, as well as at least one CAD finding corresponding to a detected region of interest in the first image. The processor determines at least one candidate location in the second image that matches the CAD finding in the first image. The matching is performed based on local appearance features extracted for the CAD finding and the candidate location. In accordance with another aspect, the processor receives digitized training images representative of at least first and second views of one or more regions of interest. Feature selection is performed based on the training images to select a subset of relevant local appearance features to represent instances in the first and second views. A distance metric is then learned based on the subset of local appearance features. The distance metric may be used to perform matching of the regions of interest.