A method and system for videoconferencing between parties at N sites. Specifically, a virtual space arrangement of N sites is created. Video sequences of N−1 remote sites are simultaneously displayed on a non-planar display arranged around a local party at a local site consistent with the virtual space arrangement. The displayed images of the remote parties are scaled consistently with their true physical sizes at their distances from the local party in the virtual space arrangement. Real-time video streams of the local party are captured from a plurality of viewpoints and are used to generate N−1 video view streams representing views of the local party from approximately the local display locations of the N−1 remote parties.