The present invention relates to the field of video encoding. In particular, it relates to a method 300 of encoding images captured by a camera and to an image processing device. An image sequence captured with an image sensor of the camera is obtained S310, and an oscillation frequency of a periodic movement of the camera during capturing of the image sequence is determined S320. A base subset of images of the image sequence corresponding to the oscillation frequency is identified S330 and the base subset of images are encoded S340 into an encoded video stream comprising intra frames and inter frames.