A video processing method includes generating a first down-sampled block by down-sampling a first block included in a first frame; generating a plurality of second down-sampled blocks by down-sampling a second block included in a second frame; selecting one of the plurality of second down-sampled blocks by comparing the first down-sampled block and each of the plurality of second down-sampled blocks; and determining, based on the first down-sampled block and the one selected second down-sampled block, a motion vector whereby the second block points to the first block, wherein the plurality of second down-sampled blocks are generated using down-sampling methods that are different from each other with respect to samples of the second block.