Systems and methods for model-based video fixing are disclosed. A video can be retrieved and analyzed to determine if any portion of the video can be represented by a model. If a portion that can be modeled is identified, a model that approximates the portion can be specified, the portion can be removed from the video, and instructions for modeling the video can be formatted. The video and the instructions can be transmitted to a receiving device, which can synthesize the model and the received video to generate a model-based video. Systems for providing the model-based video fixing are also disclosed.