In an image processing apparatus, a plurality of substantially corresponding points are designated on a reference image and a peripheral image. The degree of coincidence is detected between a window in the peripheral image with a corresponding point as the center, and windows successively cut from the reference image, whose center is a corresponding point. The corresponding point at the center of the window having the highest degree of coincidence in the reference image is moved and an offset between this corresponding point and the corresponding point on the peripheral image is detected. When a corresponding point whose offset is greater than a threshold exists, the peripheral image is subjected to an affine transformation using a corresponding point on the reference image whose offset is less than the threshold. The processing is repeated by using the corresponding point on the transformed peripheral image. The two images are merged by an affine transformation when the offsets of all corresponding points become smaller than the threshold.