An image-based object tracking system including a photographic device and a computing equipment is provided. The photographic device captures a first image of a scene at a first time and a second image of the scene at a second time subsequent to the first time. The computing equipment determines an area of the scene in the first and second images, which includes a midline of the scene in the first and second images, overlaps the first and second images for determining whether a distance between a first object in the first image and a second object in the second image is less than a predetermined threshold in response to the first object being in the area and the second object not being in the area, and updates an entry object count or an exit object count in response to the distance being less than the predetermined threshold.