A decoder for generating an audio output signal comprising one or more audio output channels from a downmix signal is provided. The downmix signal encodes one or more audio object signals. The decoder comprises a control unit (181) for setting an activation indication to an activation state depending on a signal property of at least one of the one or more audio object signals. Moreover, the decoder comprises a first analysis module (182) for transforming the downmix signal to obtain a first transformed downmix comprising a plurality of first subband channels. Furthermore, the decoder comprises a second analysis module (183) for generating, when the activation indication is set to the activation state, a second transformed downmix by transforming at least one of the first subband channels to obtain a plurality of second subband channels, wherein the second transformed downmix comprises the first subband channels which have not been transformed by the second analysis module and the second subband channels. Moreover, the decoder comprises an un-mixing unit (184), wherein the un-mixing unit (184) is configured to un-mix the second transformed downmix, when the activation indication is set to the activation state, based on parametric side information on the one or more audio object signals to obtain the audio output signal, and to un-mix the first transformed downmix, when the activation indication is not set to the activation state, based on the parametric side information on the one or more audio object signals to obtain the audio output signal. Furthermore, an encoder is provided.