The similarity calculation device is provided with a displacement vector estimation unit 101 for estimating a displacement vector between a first local region set in a first image, and a second local region which is most similar to the first local region in a second image, a geometric transformation parameter estimation unit 102 for estimating a geometric transformation parameter which geometrically transforms the first image into the second image, a displacement vector correction unit 103 for correcting the displacement vector based on the geometric transformation parameter, a scoring unit 104 for scoring the displacement vector, corrected by the displacement vector correction unit 103, in a two-dimensional space and creating a scored image, a peak detection unit 105 for detecting a peak in the scored image, and a similarity calculation unit 106 for calculating a similarity of the first image and the second image according to a size of the peak.