There is provided an image processing apparatus for applying an adjusting process to an image including multicolor image signals. An image area including a pixel to be processed is extracted, and one representative signal value is generated from signal values corresponding to a plurality of colors of pixels included in the image area (S2501). A feature amount associated with the image area is calculated based on the representative signal value (S2503, S2505, S2511), and a substitute candidate pixel common to the plurality of colors is selected from the image area based on the calculated feature amount (S2512). Signal values of the pixel to be processed are substituted by new pixel values, which are calculated from the signal values of the pixel to be processed and signal values of the substitute candidate pixel.