The present document relates to audio communication systems. In particular, the present document relates to the control of the level of audio signals within audio communication systems. A method for leveling a near-end audio signal (211) using a leveling gain (214) is described. The near-end audio signal (211) comprises a sequence of segments, wherein the sequence of segments comprises a current segment and one or more preceding segments. The method comprises determining a nuisance measure (416) which is indicative of an amount of aberrant voice activity within the sequence of segments of the near-end audio signal (211); and determining the leveling gain (214) for the current segment of the near-end audio signal (211), at least based on the leveling gain (214) for the one or more preceding segments of the near-end audio signal (211), and by taking into account—according to a variable degree—an estimate of the level of the current segment of the near-end audio signal (211); wherein the variable degree is dependent on the nuisance measure (416).