An object tracking device includes an object detection unit and a matching unit. The object detection unit includes a plurality of detectors having different processing speeds or accuracy. The object detection unit detects, for each of frames of image data input in time series, each object from the image data using one detector selected from the plurality of the detectors. The matching unit matches a current object detection result which the object detection unit generates from image data of a current frame with a previous object detection result which the object detection unit generates from image data of a previous frame before the current frame, and generates a tracking result of the object based on a matching result.