An image processing apparatus includes a plurality of dictionaries configured to store a feature of an object and information on an imaging direction in a scene for each kind of imaged scene, a detecting unit configured to detect an object with reference to at least one of the plurality of dictionaries in the scene in which the object has been imaged and which is to be learned, an estimating unit configured to estimate the imaging direction the detected object, a selecting unit configured to select one dictionary from the plurality of dictionaries based on the imaging direction estimated by the estimating unit and the information on the imaging direction in each of the plurality of dictionaries, and a learning unit configured to learn the dictionary selected by the selecting unit, based on a detection result produced by the detecting unit.