An image processing apparatus includes a frequency information extracting unit configured to extract frequency information of an input image, a color information extracting unit configured to extract color information of the input image, a feature amount calculating unit configured to calculate a feature amount of the input image on the basis of the frequency information extracted by the frequency information extracting unit and the color information extracted by the color information extracting unit, and a scene determining unit configured to determine a scene of the input image on the basis of the feature amount calculated by the feature amount calculating unit.