A method and system for identifying an unwanted caller using a call answering system. The method comprises receiving, by a call records server, information about callers from a plurality of telephony devices and one or more phone number data sources, classifying, by the call records server, one or more of the callers as either wanted or unwanted callers based on the received information, and storing, by the call records server, the received information and the classification of the one or more of the callers into a directory. The method further comprises identifying, by a call answering server, that a call from a given caller from a phone number to a receiving telephony device is from an unwanted caller by searching the directory, and handling, by the call answering server, the call from the given caller according to one or more preferences of the user, the one or more preferences including rejecting the call from the receiving telephony device, playing a recorded message, and disposing of the call.