Techniques and computing devices are described that automatically caption content directly from audio data being output from content sources, unlike other captioning systems which often rely on information contained in audio signals being sent to speakers. The disclosed techniques and computing devices may analyze metadata to determine whether the audio data is suitable for captioning or whether the audio data is some other type of audio data. Responsive to identifying audio data for captioning, the disclosed techniques and computing devices can generate a description of audible sounds interpreted from the audio data, providing for the automatic captioning of content and making audible content accessible to many users who have difficulty hearing or are otherise unable to listen to content.