A personal audio device, such as a wireless telephone, includes noise canceling circuit that adaptively generates an anti-noise signal from a reference microphone signal and injects the anti-noise signal into the speaker or other transducer output to cause cancellation of ambient audio sounds. An error microphone may also be provided proximate the speaker to measure the output of the transducer in order to control the adaptation of the anti-noise signal and to estimate an electro-acoustical path from the noise canceling circuit through the transducer. A processing circuit that performs the adaptive noise canceling (ANC) function also either adjusts the frequency response of the anti-noise signal with respect to the reference microphone signal, and/or by adjusting the response of the adaptive filter independent of the adaptation provided by the reference microphone signal.