In a computerized method, a moving object is detected in a sequence of frames of a video of a scene. Each of the frames includes a plurality of pixels representing measured light intensity values at specific locations in the scene. The pixels are organized in a regularized pattern in a memory. The object is modeled as a branched kinematic chain composed of links connected at joints. The frames are iteratively segmented by assigning groups of pixels having like pixel motion to individual links, while estimating motion parameters for the groups of pixels assigned to the individual links until the segmented pixels and their motion parameters converge and can be identified with the moving object as modeled by the kinematic chain.