Various arrangements for modeling a scene are presented. A plurality of images of the scene captured over a period of time may be received, each image comprising a plurality of pixels. A plurality of background models may be created using the plurality of images. At least one background model may be created for each pixel of the plurality of pixels. A plurality of foreground models may be created using the plurality of images. A foreground model may be created for each pixel of at least a first subset of pixels of the plurality of pixels. The background models and the foreground models may be indicative of the scene over the period of time.