Video-based object tracking accuracy is improved by a tentative identification of objects to be tracked. An identified blob that does not encompass a previously established set of tracking features (“tracker”) triggers initialization of an infant tracker. If that tracker remains the only tracker or becomes the “oldest” tracker associated with a blob identified in subsequent video frames, the “age” of the tracker is increased. If, in subsequent frames, the tracker is encompassed by a blob that is associated with an “older” tracker, the “age” of the tracker is decreased. Infant trackers that reach or exceed a threshold “age” are promoted to adult status. Adult trackers can be processed as being associated with valid objects. Trackers established for blobs identified due to mask segmentation tend not to cause false object detections. When segmentation is corrected, blob segments are combined and redundant trackers for the associated object are demoted and ignored.