According to an embodiment, a method for filtering descriptors for visual object recognition is provided. The method includes identifying false positive descriptors having a local match confidence that exceeds a predetermined threshold and a global image match confidence that is less than a second threshold. The method also includes training at least one classifier to discriminate between the false positive descriptors and other descriptors. The method further includes filtering feature point matches using the at least one classifier. According to another embodiment, the filtering step may further include removing one or more feature point matches from a result set. According to a further embodiment, a system for filtering feature point matches for visual object recognition is provided. The system includes a hard false positive identifier, a classifier trainer and a hard false positive filter.