Devices, systems, methods, media, and programs for detecting an emotional state change in an audio signal are provided. A number of segments of the audio signal are analyzed based on separate lexical and acoustic evaluations, and, for each segment, an emotional state and a confidence score of the emotional state are determined. A current emotional state of the audio signal is tracked for each of the number of segments. For a particular segment, it is determined whether the current emotional state of the audio signal changes to another emotional state based on the emotional state and a comparison of the confidence score of the particular segment to a predetermined threshold.