A method by which a tracking apparatus tracks a target object includes: acquiring a first tree structure indicating a tracking processing order of frames, each frame including a tracking area in which the target object is located; acquiring a plurality of frame groups, each frame group consisting of two frames, and acquiring distance evaluation values of the respective frame groups; acquiring a second tree structure based on the first tree structure and the distance evaluation values; and tracking the target object based on the acquired second tree structure, wherein the distance evaluation value is determined based on at least one of locations of tracking areas included in two frames belonging to the frame group and pixel values included in the tracking areas.