A method of and apparatus to efficiently encode and/or decode an image are provided. The method of encoding an image includes: calculating a motion vector of a main reference image by estimating a motion of a current image with reference to the main reference image; deriving a motion vector of at least one auxiliary reference image from this motion vector; generating a prediction image of the current image by using a motion vector of the main reference image or the motion vector of the at least one auxiliary reference image; and encoding a residue image between the current image and the prediction image.