Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing contextual information for presented media. In one aspect, a method includes storing in a buffer, on a first user device, media data as buffered media data, the buffered media data being a most recent portion of media data received at the first user device, the most recent portion inclusive of the media data received from a present time to a prior time that is fixed relative to the present time; responsive to a search operation invocation at the present time, sending the buffered media data to a search processing system that is remote from the first user device; and receiving, from the search processing system and in response to the buffered media data, contextual information regarding an entity that the data processing system identified from processing the buffered media data.