The present invention provides a motion compensating apparatus which is capable of enhancing the overall coding efficiency by taking the amount of coded information used to code a motion vector as well as the sum of difference absolute values of estimated differences into consideration in order to determine an optimal vector. The motion compensating apparatus includes: a motion compensation processing unit for outputting a motion vector between an input image and a reference image, and an estimated image which is extracted from the reference image in accordance with the motion vector; a unit for calculating the sum of difference absolute values, as a distortion amount calculating unit, to obtain a distortion amount between the input image and the estimated image; a vector value coding unit for receiving the motion vector to code the motion vector thus inputted thereto to output a vector coded amount; and an optimal vector determining unit for receiving the motion vector, the distortion amount and the vector coded amount to obtain, for all of a plurality of motion vectors to be evaluated, an evaluation function calculated from the distortion amount and the vector coded amount to output as an optimal vector the motion vector in which the evaluation function exhibits a minimum value.