The invention relates to an audio signal processing apparatus for processing an input earpiece audio signal upon the basis of a microphone audio signal, the audio signal processing apparatus comprising a voice activity detector being configured to determine a voice activity indicator signal upon the basis of the input earpiece audio signal, a noise magnitude determiner being configured to determine a microphone noise magnitude indicator signal upon the basis of the microphone audio signal, a gain factor determiner being configured to determine a gain factor signal upon the basis of the voice activity indicator signal and the microphone noise magnitude indicator signal, and a weighter being configured to weight the input earpiece audio signal by the gain factor signal to obtain an output earpiece audio signal.