Disclosed are a video processing method and a device therefor. The video processing method may include receiving a video comprising a plurality of temporal portions, receiving a first model parameter corresponding to a first neural network to process the video entirely, receiving residues between the first model parameter and a plurality of second model parameters corresponding to a plurality of second neural networks to process the plurality of temporal portions, and performing at least one of super-resolution, reverse or inverse tone mapping, tone mapping, frame interpolation, motion deblurring, denoising, and compression artifact removal on the video based on the residues.