An apparatus for generating a sound field description from an input signal comprising one or more channels, comprises: an input signal analyzer (600) for obtaining diffuseness data from the input signal; a sound component generator (650) for generating, from the input signal, one or more sound field components of a first group of sound field components having for each sound field component a direct component and a diffuse component, and for generating, from the input signal, a second group of sound field components having only a direct component, wherein the sound component generator is configured to perform an energy compensation when generating the first group of sound field components, the energy compensation depending on the diffuseness data and at least one of a number of diffuse components in the first group, and a maximum order of sound field components of the second group.