To improve precision of visual search processing, SIFT points within a query image are forward matched to features in each of a plurality of repository images and SIFT points within each repository image are backward matched to features within the query image. Forward-only, backward-only and forward-and-backward matches may be weighted differently in determining an image match. Two way matching may be triggered by query image bit rate in excess of a threshold or by a sum of weighted distances between matching points exceeding a threshold. Significant performance gains in eliminating false positive matches are achieved.