An audio encoder comprises a multi-channel receiver which receives an M-channel audio signal where M>2. A down-mix processor down-mixes the M-channel audio signal to a first stereo signal and associated parametric data and a spatial processor modifies the first stereo signal to generate a second stereo signal in response to the associated parametric data and spatial parameter data for a binaural perceptual transfer function, such as a Head Related Transfer Function (HRTF). The second stereo signal is a binaural signal and may specifically be a (3D) virtual spatial signal. An output data stream comprising the encoded data and the associated parametric data is generated by an encode processor and an output processor. The HRTF processing may allow the generation of a (3D) virtual spatial signal by conventional stereo decoders. A multi-channel decoder may reverse the process of the spatial processor to generate an improved quality multi-channel signal.