A binarization processing unit 11 performs a binarization process the basis of existence of at least one of lightness and chroma for a candidate image of template matching. A labeling processing unit 12 performs a labeling process for the binarized candidate image and thereby extracts a pixel block in the binarized candidate image. An outline determining unit 14 performs sampling of a position on an outermost periphery of the pixel block, and identifies a range from a centroid position of the pixel block and an angle from a predetermined direction as the position. A correlation determining unit 15 determines a correlation between the ranges and the angles of both the candidate image and the reference image on the basis of the reference outline data, and excludes the candidate image from a candidate of the template matching in accordance with degree of the correlation.