Disclosed is an image processing device configured to obtain a plurality of difference maps between a first frame or a first feature map corresponding to the first frame, and a plurality of second feature maps corresponding to a second frame, obtain a plurality of third feature maps and a plurality of fourth feature maps by performing pooling processes on the plurality of difference maps according to a first size and a second size, obtain a plurality of modified difference maps by weighted-summing the plurality of third feature maps and the plurality of fourth feature maps, identify any one collocated sample by considering sizes of sample values of collocated samples of the plurality of modified difference maps corresponding to a current sample of the first frame, and determine a filter kernel used to obtain the second feature map corresponding to the modified difference map including the identified collocated sample, as a motion vector of the current sample.