To interpolate a frame between two frames of a video signal, sets of reference images are generated, each set having a different resolution level. Motion between the two frames is estimated at each resolution level by using these sets of reference images. For each pixel processed at each resolution level, multiple motion vector candidates are obtained. Information indicating the multiple motion vector candidates is used to select motion search ranges at the next higher resolution level. To determine the motion search range for a pixel, selected motion vector candidates pertaining both to the pixel itself and to its neighboring pixels are used. An interpolated frame of high image quality is thereby obtainable without increased computation and with reduced risk of major image defects due to erroneous motion estimation.