An image processing device configured to obtain difference maps between a first frame or a first feature map corresponding to the first frame, and second feature maps corresponding to a second frame, obtain third feature maps and fourth feature maps by performing pooling processes on the difference maps according to a first size and a second size, obtain modified difference maps by weighted-summing the third feature maps and the fourth feature maps, identify any one collocated sample based on sizes of sample values of collocated samples of the modified difference maps corresponding to a current sample of the first frame, and determine a filter kernel used to obtain the second feature map corresponding to the modified difference map including the identified collocated sample, as a motion vector of the current sample.