A system and method for local deformable motion analysis and for accurately tracking motion of an object isolating local motion of an object from global motion of an object is disclosed. The object is viewed in an image sequence and image regions are sampled to identify object image regions and background image regions. The motion of at least one of the identified background image regions is estimated to identify those background image regions affected by global motion. Motion from multiple background image regions are combined to measure the global motion in that image frame. The measured global motion in the object image regions are compensated to measure local motion of the object and the local motion of the object is tracked. A system and method for accurately measuring the local deformable motion of an object as the relative motion between two control point sets is disclosed. The point sets are defined as the inner contour and the outer contour of an object. The motion of the control point sets is estimated and the relative motion is used to characterize the local deformation and local motion of the object.