A movement vector detecting apparatus for detecting a movement vector of an object between frames which are close to each other on a block unit basis is provided. A first mode to detect the movement vector by using color components and a second mode to detect the movement vector by using a luminance component are switched in accordance with chromaticity statistics of an input block, thereby detecting the movement vector. There is also provided an encoding apparatus wherein a difference between an input signal to be encoded and a prediction value is quantized, the quantized signal is generated, the quantized differential signal is inversely quantized the sum of the inversely quantized reproduction differential signal and the prediction value is stored, the stored signal is read out on the basis of an address signal regarding the input signal, the read-out signal and the input signal are received as inputs and the prediction value is obtained, and when the prediction value is derived, an evaluation function to obtain the prediction value is determined in accordance with chromaticity statistics of the input signal.