For the purpose of providing an image encoding technique for reducing the amount of codes, in inter prediction in which motion search is performed by using a block obtained by dividing an input image and a reference image, the motion vector of the block is predicted by selecting from the motion vectors of the surrounding blocks. Further, in the method of calculating a differential vector and then performing encoding, the number or positions of candidate blocks from which a prediction vector is selected are changed block by block or frame by frame and thereby adaptive and efficient video image compression is performed.