Disclosed are a system and method for quantifying changes in animal posture from video recordings of multiple interacting animals. Also disclosed are a system and method for training a posture identification algorithm efficiently with minimal human effort and evaluating the posture of millions of animal images. The disclosed process employs video recordings of behaving animals to produce a full set of geometrical coordinates for each body part over time. Inter alia, the disclosed system should be useful for organizations interested in doing large small-molecule screens to find drugs that affect the behavior of animals, and for those investigating new behavioral diagnostics for human patients. The disclosed system allows one to record video of animals and then automatically calculate the position of each body part over time. The output of this calculation can be used for simple analysis of the amount of movement for specific body parts, or part of a more complex scheme for measuring behaviors.