An apparatus for downsampling an audio signal comprise a first filter bank (102, 601, 2302) for generating a plurality of subband signals from the audio signal, wherein a sampling rate of the subband signals is smaller than a sampling rate of the audio signal; at least one synthesis filter bank (602-2, 2304) followed by an analysis filter bank (603-2, 2307) for performing a sample rate conversion to obtain a sample rate converted signal, the synthesis filter bank (602-2) having a number of channels different from a number of channels of the analysis filter bank (603-2); a time stretch processor (604-2, 2309) for processing the sample rate converted signal to obtain a time stretched signal; and a combiner (605, 2311) for combining the time stretched signal and a low-band signal or a different time stretched signal.