A conference controller (111, 175) configured to place an upstream audio signal (123, 173) associated with a conference participant and a sound signal within a 2D or 3D conference scene to be rendered to a listener (211) is described. The conference controller (111, 175) is configured to set up a X-point conference scene with X different spatial talker locations (212) within the conference scene, X being an integer, X>0; assign the upstream audio signal (123, 173) to one of the talker locations (212); place a sound signal at a spatial sound location (503) within the X-point conference scene; and generate metadata identifying the assigned talker location (212) and the spatial sound location and enabling an audio processing unit (121, 171) to generate a spatialized audio signal based on a set of downstream audio signals (124, 174) comprising the upstream audio signal (123, 173) and the sound signal.