An apparatus and method for echo cancellation is presented. The echo canceller comprises an adaptive filter that tracks the impulse response of the echo path and produces an estimate of the echo. Filter adaptation is controlled by a controller based on the rate of the far-end speech signal, the rate of the near-end signal, an acoustic loss measure, and a double talk hangover indicator. The controller may also comprise a step size adaptation unit for determining the adaptation step size of the adaptive filter. In addition, the controller may comprise a noise replacement unit, which controls replacement of the echo residual signal with comfort noise to ensure echo is completely rejected when only the far-end speaker is talking.