Audio signal processing apparatuses and methods are provided, such as an audio signal downmixing apparatus for processing an input audio signal into an output audio signal, wherein the input audio signal comprises a plurality of input channels recorded at a plurality of spatial positions and the output audio signal comprises a plurality of primary output channels. The audio signal downmixing apparatus comprises a downmix matrix determiner configured to determine for each frequency bin j of a plurality of frequency bins a downmix matrix DU with j being an integer in the range from 1 to N, and a processor configured to process the input audio signal using the downmix matrix DU into the output audio signal.