A personal audio device, such as a wireless telephone, generates an anti-noise signal from an error microphone signal and injects the anti-noise signal into the speaker or other transducer output to cause cancellation of ambient audio sounds. The error microphone is also provided proximate the speaker to provide an error signal indicative of the effectiveness of the noise cancellation. A secondary path estimating adaptive filter is used to estimate the electro-acoustical path from the noise canceling circuit through the transducer so that source audio can be removed from the error signal. Noise bursts are injected intermittently and the adaptation of the secondary path estimating adaptive filter controlled, so that the secondary path estimate can be maintained irrespective of the presence and amplitude of the source audio.