A system for enhancing the sound signal produced by an audio system in a listening environment by compensating for ambient noise in the listening environment is provided. The system receives an electrical sound signal and generates a sound output therefrom. A total sound signal is sensed representative of the total sound level in the environment, where the total sound level includes both the sound output from the audio system and the ambient noise within the environment. The system extracts an ambient noise signal representative of the ambient noise in the environment from the total sound signal in response to the total sound signal and to a reference signal derived from the electrical sound signal. The system extracts the ambient noise signal using an adaptive filter with an adaptive step size. The system generates a control signal in response to the ambient noise signal and adjusts the sound output of the audio system to compensate for the ambient noise level in response to the control signal. The system calculates a step size for controlling the adaptive step size of the adaptive filter.