In accordance with the present disclosure, an integrated circuit for implementing at least a portion of a personal audio device may include an output and a processing circuit. The output may provide an output signal to a transducer including both a source audio signal for playback to a listener and an anti-noise signal for countering the effect of ambient audio sounds in an acoustic output of the transducer. The processing circuit may implement an adaptive noise cancellation system that generates the anti-noise signal to reduce the presence of the ambient audio sounds heard by the listener by adapting, based on a presence of the source audio signal, a response of the adaptive noise cancellation system to minimize the ambient audio sounds at the acoustic output of the transducer, wherein the adaptive noise cancellation system is configured to adapt both in the presence and the absence of the source audio signal.