Methods, apparatuses, and computer-readable media are provided for maintaining blob trackers for video frames. For example, a first blob tracker maintained for a current video frame is identified. The first blob tracker is associated with a blob detected in one or more video frames. The blob includes pixels of at least a portion of a foreground object in the one or more video frames. It is determined that the first blob tracker is a first type of tracker. Trackers having the first type are associated with objects that have transitioned at least partially into a background model (referred to as sleeping objects and sleeping trackers). One or more interactions are identified between the first blob tracker and at least one other blob tracker. The at least one other blob tracker can be the first type of tracker or can be a second type of tracker that is not a sleeping tracker (the second type of tracker is not associated with an object that has transitioned at least partially into the background model. A characteristic of the first blob tracker can then be modified based on the identified one or more interactions. Modifying the characteristic of the first blob tracker can include transitioning the first blob tracker from the first type of tracker to the second type of tracker, updating an appearance model of the first blob tracker, and/or other suitable characteristic of the first blob tracker.