An embodiment of a semiconductor package apparatus may include technology to process a single stream of video frames which includes frame information from two or more video sources, and process a current frame based on information from a prior frame in the single stream of video frames which precedes the current frame by a number of frames based on a number of video sources represented in the single stream of video frames. Other embodiments are disclosed and claimed.