A system and method are provided. The system includes an image capture device configured to capture a video sequence formed from a set of input image frames and including a set of objects. The system further includes a processor configured to detect the objects to form object detections and track the object detections over the input frames to form tracked detections. The processor is also configured to generate for a current input frame, responsive to conditions, a set of sparse object proposals for a current location of an object based on: (i) the tracked detections of the object from an immediately previous input frame; and (ii) detection proposals for the object derived from the current frame. The processor is additionally configured to provide a user perceptible indication of the current location of the object, based on the set of sparse object proposals.