A contact center system for performing call progress analysis including tone and speech classification according to an embodiment includes at least one processor and at least one memory comprising a plurality of instructions stored thereon that, in response to execution by the at least one processor, causes the contact center system to determine a cumulative average of an entropy of an audio signal received by the contact center system, determine a cumulative average power spectral amplitude and a cumulative average spectral entropy of the audio signal, calculate a difference measure of the audio signal as a difference between the cumulative average of the entropy and the cumulative average spectral entropy, distinguish tones from speech of the audio signal based on the difference measure, and process one or more tones of the audio signal in response to identification of the one or more tones in the audio signal.