A method that generates binaural headphone playback signals given multiple audio source signals with an associated metadata and binaural room impulse response (BRIR) database, wherein the audio source signals are channel-based, object-based, or a mixture of both channel-based and object-based signals. The method includes parameterizing BRIR to be used for rendering, dividing each audio source signal to be rendered into a number of blocks and frames, and averaging the parameterized BRIR sequences. The method also includes downmixing the divided audio source signals using the diffuse blocks of BRIRs, and performing late reverberation processing on the downmixed version of the previous blocks of the audio source signals.