An image processing apparatus comprises: a separation unit configured to separate image data into a luminance signal and a color difference signal; a decision unit configured to decide a reference pixel, which is referenced for a pixel of interest included in the image data; a luminance noise reduction unit configured to reduce luminance noise in the luminance signal of the image data; a weight calculation unit configured to calculate a weight of the reference pixel based on similarity of luminance included in the luminance signal having been subjected to noise reduction by the luminance noise reduction unit between a first area containing the pixel of interest and a second area containing the reference pixel; and a color noise reduction unit configured to reduce color noise in the color difference signal for the pixel of interest by using the weight of the reference pixel.