An image processing apparatus includes: a unit producing a histogram in a color space from an input image; a unit extracting a plurality of peak colors having a local maximum frequency of appearance of a color value in the histogram; a unit determining whether to unify the extracted plurality of peak colors, based on a feature amount of the extracted plurality of peak colors, and selecting a peak color as a representative color when the extracted plurality of peak colors are unified, the feature amount being a lightness difference among the extracted plurality of peak colors and directions of vectors in the color space, the vectors each showing a shortest distance from one of the extracted plurality of peak colors to a line segment connecting between a reference color and a dark color; and a unit replacing a color of each pixel in the input image by the representative color.