An apparatus configured to estimate a position of a part of an object in an image includes: an image receiver configured to receive the image; a reference point setter configured to set a reference point in the image; a controller configured to generate information about the reference point by repeating a process a predetermined number of times, the process comprising obtaining one piece of direction information about a probability and a direction that the reference point is to be moved to the part of the object by a classifier, and resetting the reference point by moving the reference point a predetermined distance based on the one piece of direction information; and a location estimator configured to estimate a position of the part of the object in the image by using the information about the reference point as the reference point is reset the predetermined number of times.