The present invention relates to the performance of motion vector refinement in a search space for multi-reference inter-prediction. Two or more reference pictures are selected, one of those used for motion vector refinement. Based on an initial estimate of a motion vector to the reference picture for motion vector refinement, a search space in this reference image is constructed. Using template matching, the first motion vector is refined. The second motion vector to another reference picture is calculated using its initial estimate, the initial estimate of the first motion vector and the refined first motion vector.