Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for recognizing names of entities in speech. In one aspect, a method includes actions of receiving an utterance that includes (i) a first term that indicates a particular entity type, and (ii) a second term that indicates an entity name. Additional actions include obtaining a phonetic representation of the second term and determining that the phonetic representation of the second term matches a particular phonetic representation of a particular canonical name of a set of canonical names associated with a particular entity. Further actions include outputting a reference name associated with the particular entity as a transcription of the second term.