Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving data that represents actions taken by a particular user with respect to entities of a plurality of entity types, disambiguating the data by identifying entities specified in the data using a machine learning based annotator that is trained to recognize entities and annotating the data, receiving a query specific from the particular user, querying the data that represents actions taken by the particular user, generating an answer to the query from the data representing the actions taken by the particular user, and providing the answer to the query.