A method for propagating user-provided foreground-background constraint information for a first video frame to subsequent frames allows extraction of moving foreground objects with minimal user interaction. Video matting is performed wherein constraints derived from user input with respect to a first frame are propagated to subsequent frames using the estimated alpha matte of each frame. The matte of a frame is processed in order to arrive at a rough foreground-background segmentation which is then used for estimating the matte of the next frame. At each frame, the propagated constraints are used by an image matting method for estimating the corresponding matte which is in turn used for propagating the constraints to the next frame, and so on.