An audio encoder for encoding segments of coefficients, the segments of coefficients representing different time or frequency resolutions of a sampled audio signal, the audio encoder including a processor for deriving a coding context for a currently encoded coefficient of a current segment based on a previously encoded coefficient of a previous segment, the previously encoded coefficient representing a different time or frequency resolution than the currently encoded coefficient. The audio encoder further includes an entropy encoder for encoding entropy the current coefficient based on the coding context to obtain an encoded audio stream.