A digest generation system obtains video streams and includes an admission control module that selects, for each video stream, a subset of the frames of the video stream to analyze. A frame-to-text classifier generates a digest for each selected frame and the generated digests are stored in a digest store in a manner so that each digest is associated with the video stream from which the digest was generated. The digest for a frame is text that describes the frame, such as objects identified in the frame. A viewer desiring to view a video stream having particular characteristics inputs a text search query to a search system. The search system, based on the digests, generates search results that are an indication of video streams that satisfy the search criteria. The search results are presented to the user, allowing the user to select and view one of the video streams.