Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting labels for a non-text content item. In one aspect, a method receives a set of initial labels for a non-text content item, wherein the set of initial labels specifies text that has been identified as descriptive of the non-text content item and a web page to which the text corresponds. Initial labels corresponding to sets of matching web pages are grouped into separate initial label groups that correspond to each set of matching web pages. Sets of matching labels are grouped into other separate initial label groups that correspond to the sets of matching labels. One or more words that are included in at least a threshold number of the separate label groups are selected as final labels for the non-text content item.