The invention discloses a skeleton tracking-based dynamic real-time three-dimensional reconstruction method and a skeleton tracking-based dynamic real-time three-dimensional reconstruction system. Themethod comprises the following steps: performing depth image shooting on a human body to acquire a single depth image; converting the single depth image into three-dimensional point cloud and acquiring a matched point pair between the three-dimensional point cloud and a reconstruction model vertex; establishing an energy function according to the matched points and jointly solving a non-rigid motion position conversion parameter and a human body skeleton motion parameter on each vertex on the reconstruction model; and solving the energy function, aligning the reconstruction model with the three-dimensional point cloud according to the solving result, and updating and complementing the aligned model by using the depth image to realize three-dimensional human body reconstruction. By the method, the human body can be shot by a depth image, so that the depth image is acquired to serve as system input information; furthermore, the function of performing real-time three-dimensional reconstruction is completed based on the depth image, and the method and the system are accurate and robust in solution and are simple and easy to realize.