To detect events in an audio stream, frames of an audio signal (e.g., frames generated by a codec for a voice call or music stream) are received. Based on information in the frames, an index is used to look up an entry in a table associated with the codec. Each entry in the table indicates a likelihood that a frame matches a sound model element. The likelihood is used in the search for a sound bite, word, and/or phrase in the audio signal. The process of dynamic programming is used to find the combined likelihood for a match of the word, phrase, and/or sound bite to a region of the audio stream. Upon detection of the word, phrase, and/or sound bite in the audio stream, an event is generated, such as, notifying a person or logging the event in a database.