An image processing apparatus, which correctly extracts specular reflected light components of reflected light from an object, and accurately estimates a light source color, is provided. The image processing apparatus calculates a pixel value difference distribution by repeating, for respective pixels, to calculate pixel value differences between a pixel of interest and adjacent pixels in an input image and to calculate similarities between pixel value differences. A light source estimation unit estimates a color of a light source which illuminates an object in the input image based on the calculated distribution.