This disclosure presents methods, systems, computer-readable media, and apparatuses for optically tracking the location of one or more objects. The techniques may involve accumulation of initial image data, establishment of a dataset library containing image features, and tracking using a plurality of modules or trackers, for example an optical flow module, decision forest module, and color tracking module. Tracking outputs from the optical flow, decision forest and/or color tracking modules are synthesized to provide a final tracking output. The dataset library may be updated in the process.