An image processing apparatus includes a comparison unit that selects a pixel of interest in a processing image, and compare magnitudes of luminance value of the pixel of interest and luminance value of each of a plurality of neighboring pixels having a predetermined positional relationship with the pixel of interest; a calculation unit that calculates a feature amount of the pixel of interest based on the predetermined positional relationship between the pixel of interest and each of the plurality of neighboring pixels and a comparison result obtained by the comparison unit. For two neighboring pixels at positions which are point symmetrical with respect to the pixel of interest, the comparison unit sets that only one of the two neighboring pixels has the predetermined positional relationship.