In general, in one aspect, a method includes receiving, by one or more processors, a query image, obtaining, by the one or more processors, search results responsive to the query image, each search result identifying a corresponding web page containing an image that is determined to be responsive to the query image, the search results being ordered in a first arrangement, obtaining, by the one or more processors, text labels associated with the query image, and generating, by the one or more processors a second arrangement of the search results by ordering the search results based on a presence in the respective corresponding web pages of one or more of the text labels associated with the query image. This and other implementations of this aspect can include corresponding systems, apparatus, and computer program products.