A system and method are described for performing motion capture on a subject. For example, a computer-implemented method according to one embodiment of the invention comprise: creating a scalar field for the three-dimensional (3-D) capture volume of the subject; generating a surface mesh for the scalar field; retaining good vertices and removing bad vertices of the surface mesh; and storing the good vertices for use in subsequent reconstruction of the motion of the subject. Another computer-implemented method comprises: capturing a series of image frames of the subject over a period of time each frame each frame having a plurality of vertices defining a captured surface of the subject; establishing a reference frame having one or more of the plurality of vertices; performing frame-to-frame tracking to identify vertices within the N′th frame based on the (N−1)′th frame or an earlier frame; and performing reference-to-frame tracking to identify vertices within the N′th frame based on the reference frame to counter potential drift between the frames. Yet another computer-implemented method comprises: capturing motion capture data including a plurality of images of the N vertices during a motion capture session; retrospectively identifying X of the N vertices to track across the plurality of images where X