An audio encoder for encoding an audio signal includes an impulse extractor (10) for extracting an impulse-like portion from the audio signal. This impulse-like portion is encoded and forwarded to an output interface (22). Furthermore, the audio encoder includes a signal encoder (16) which encodes a residual signal derived from the original audio signal so that the impulse-like portion is reducedor eliminated in the residual audio signal. The output interface (22) forwards both, the encoded signals, i.e., the encoded impulse signal (12) and the encoded residual signal (20) for transmission or storage. On the decoder-side, both signal portions are separately decoded and then combined to obtain a decoded audio signal.