A method includes defining a background model of the video sequence by applying a first algorithm, the background model defining whether that spatial area belongs to a background or a foreground in the video sequence, wherein a detected significant change in image data in a spatial area in an image frame relative image data in said spatial area in a preceding image frame is indicative of said spatial area belonging to the foreground; indicating that an idle area of the defined foreground areas is to be transitioned from foreground to background; and determining whether the idle area is to be transitioned by applying a second algorithm to image data of an image frame of the video sequence, the image data at least partly corresponding to the idle area; wherein if the idle area is not to be transitioned, maintaining the idle area as a foreground area in the background model.