An image processing apparatus includes a face image detector detecting a face image from an image, a reference mask generator generating a reference mask based on the arrangement of parts included in the face image detected by the face image detector, a face color area detector detecting a face color area from the image, and a face image searcher searching for the face image using the reference mask in the face area detected by the face color area detector.