A personal audio device, such as a wireless telephone, includes noise canceling circuit that adaptively generates an anti-noise signal from a reference microphone signal and injects the anti-noise signal into the speaker or other transducer output to cause cancellation of ambient audio sounds. An error microphone is provided proximate the speaker to measure the output of the transducer in order to control the adaptation of the anti-noise signal and to estimate an electro-acoustical path from the noise canceling circuit through the transducer. The anti-noise signal is adaptively generated to minimize the ambient audio sounds at the error microphone. A processing circuit that performs the adaptive noise canceling (ANC) function also filters one or both of the reference and/or error microphone signals, to bias the adaptation of the adaptive filter in one or more frequency regions to alter a degree of the minimization of the ambient audio sounds at the error microphone.