An apparatus comprises a processor (301) providing a plurality of reference video streams for a plurality of reference viewpoints for a scene. A receiver (305) receives a viewpoint request from a remote client where the viewpoint request is indicative of a requested viewpoint. A generator (303) generates an output video stream comprising a first video stream with frames from a first reference video stream and a second video stream with frames from a second reference video stream. The frames of the second video stream are differentially encoded relative to the frames of the first video stream. A controller (307) selects the reference video stream for the first and second video streams in response to the viewpoint request and may be arranged to swap the reference video streams between being non-differentially encoded and being differentially encoded when the viewpoint request meets a criterion.