A method and apparatus of processing caller experiences is disclosed. One example method may include determining a call event type occurring during a call and assigning a weight to the call event type via a processing device. The method may also include calculating a caller experience metric value representing a caller's current call status responsive to determining the at least one call event type, the caller experience metric being a function of the current event type weight and a discounting variable that discounts a value of past events. The method may also provide comparing the caller experience metric to a predefined threshold value and determining whether to perform at least one of transferring the call to a live agent and switching from a current caller modality to a different caller modality.