There is provided a color processing apparatus that performs color reproduction in accordance with the brightness of a subject at the time of photographing. An acquisition unit acquires image data. A setting unit sets viewing environment parameters for a plurality of luminance ranges based on a luminance histogram of the image data that is acquired. A processing unit performs color conversion processing using a color appearance model, using the viewing environment parameters that are set in accordance with the luminance of pixels of the image data that is acquired.