Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing contextual information for a displayed resource that includes an image. In one aspect, a method includes providing, by a user device to a contextual information server, a query-independent request for contextual information relevant to an active resource displayed in an application environment on the user device, wherein the request specifies content of the active resource and further specifies that the active resource displayed on the user device includes an image, but does not include the image in the request, receiving a request for the image from the contextual information server, providing the image to the contextual information server, receiving a user interface element that includes contextual information regarding the image, and displaying the user interface element on the user device with the active resource.