The present invention provides a system and method for processing sequence of image frames captured through an imaging device, the imaging device includes an accelerometer. The system and method comprise an adaptive temporal background subtraction unit and a dynamic foreground object estimation unit for evaluating whether each pixel is either a moving object pixel or a non-moving object pixel based on the classification of the pixels between the first motion map and the second motion map generated thereon.