A personal audio device, such as a wireless telephone, includes an adaptive noise canceling (ANC) circuit that adaptively generates an anti-noise signal from a reference microphone signal and injects the anti-noise signal into the speaker or other transducer output to cause cancellation of ambient audio sounds. An error microphone is also provided proximate the speaker to estimate an electro-acoustical path from the noise canceling circuit through the transducer. A processing circuit determines a degree of coupling between the user's ear and the transducer and adjusts the adaptive cancellation of the ambient sounds to prevent erroneous and possibly disruptive generation of the anti-noise signal if the degree of coupling lies either below or above a range of normal operating ear contact pressure.