An adaptive noise canceller adapts a secondary path modeling response using ambient noise, rather than using another noise source or source audio as a training source. Anti-noise generated from a reference microphone signal using a first adaptive filter is used as the training signal for training the secondary path response. Ambient noise at the error microphone is removed from an error microphone signal, so that only anti-noise remains. A primary path modeling adaptive filter is used to modify the reference microphone signal to generate a source of ambient noise that is correlated with the ambient noise present at the error microphone, which is then subtracted from the error microphone signal to generate the error signal. The primary path modeling adaptive filter is previously adapted by minimizing components of the error microphone signal appearing in an output of the primary path adaptive filter while the anti-noise signal is muted.