An amount-of-movement detection device which is capable of detecting the amount of movement with a high degree of accuracy when the amount of movement of an image is detected by hierarchical block matching method. An amount-of-movement detection device 1 forms image data S2.sub.n and S3.sub.n of plural hierarchies having different resolutions from each other, by input image S1.sub.n input at a first point in time, and forms multiplexed image data S2.sub.n-1 and S3.sub.n-1 of a higher hierarchy of low resolution, based on an input image S1.sub.n-1 input at a second point in time, by selecting pixels overlapping with each other from the image data of the lower hierarchy having higher resolution and reducing them in size, in order to detect a motion vector by block matching at a predetermined block unit for each of the hierarchies corresponding to the hierarchized image data S2.sub.n and S3.sub.n and the multiplexed and hierarchized image data S2.sub.n-1 and S3.sub.n-1. Thus, the accuracy in the detection of an amount of movement can be greatly enhanced.