A method of encoding an input video including a sequence of video frames as a hybrid video stream, comprises downsampling the input video from an original spatial resolution to a reduced spatial resolution and an intermediate spatial resolution; providing the input video at the reduced spatial resolution to a base encoder to obtain a base encoded stream; providing a first enhancement stream based on first residuals at the intermediate spatial resolution; and providing a second enhancement stream based on second residuals based at the original spatial resolution, which is at least partially encoded using temporal prediction. The method further comprises detecting at least one non-motion region in a video frame, and causing the set of first residuals but not the set of second residuals to vanish throughout the non-motion region.