The systems and techniques disclosed herein provide the ability to determine locations visited by a user and associate relevant location labels with the locations visited based on contact information. In some examples, a location label can be applied based on a match between a location visited and information stored in a user's contact list. In other examples, a user can efficiently designate a contact and location label to be associated with a location visited. In still other examples, if a location visited by a user is not listed in the user's contact list, but is otherwise known to the system, the location visited can be appropriately labeled and the corresponding contact in the user's contact list can be updated to include the location visited.