An apparatus to facilitate motion tracking is disclosed. The apparatus includes one or more processors to receive video data including a plurality of point clouds in a time series, convert each of the plurality of point clouds into a deformable model representation and generate an inter-frame mapping between the deformable model representations to track motion between the plurality of point clouds.