公开(公告)号：US11238634B2

公开(公告)日：2022-02-01

申请号：US16860411

申请日：2020-04-28

Applicant: Adobe Inc.

Inventor： Jimei Yang ,  Davis Rempe ,  Bryan Russell ,  Aaron Hertzmann

IPC: G06T13/40 ,  G06T15/20 ,  G06T15/50 ,  G06N3/08

Abstract: In some embodiments, a motion model refinement system receives an input video depicting a human character and an initial motion model describing motions of individual joint points of the human character in a three-dimensional space. The motion model refinement system identifies foot joint points of the human character that are in contact with a ground plane using a trained contact estimation model. The motion model refinement system determines the ground plane based on the foot joint points and the initial motion model and constructs an optimization problem for refining the initial motion model. The optimization problem minimizes the difference between the refined motion model and the initial motion model under a set of plausibility constraints including constraints on the contact foot joint points and a time-dependent inertia tensor-based constraint. The motion model refinement system obtains the refined motion model by solving the optimization problem.

公开(公告)号：US11721056B2

公开(公告)日：2023-08-08

申请号：US17573890

申请日：2022-01-12

Applicant: Adobe Inc.

Inventor： Jimei Yang ,  Davis Rempe ,  Bryan Russell ,  Aaron Hertzmann

IPC: G06T13/40 ,  G06T15/20 ,  G06T15/50 ,  G06N3/08

CPC classification number: G06T13/40 ,  G06N3/08 ,  G06T15/20 ,  G06T15/50

Abstract: In some embodiments, a model training system obtains a set of animation models. For each of the animation models, the model training system renders the animation model to generate a sequence of video frames containing a character using a set of rendering parameters and extracts joint points of the character from each frame of the sequence of video frames. The model training system further determines, for each frame of the sequence of video frames, whether a subset of the joint points are in contact with a ground plane in a three-dimensional space and generates contact labels for the subset of the joint points. The model training system trains a contact estimation model using training data containing the joint points extracted from the sequences of video frames and the generated contact labels. The contact estimation model can be used to refine a motion model for a character.

公开(公告)号：US20210335028A1

公开(公告)日：2021-10-28

申请号：US16860411

申请日：2020-04-28

Applicant: Adobe Inc.

Inventor： Jimei Yang ,  Davis Rempe ,  Bryan Russell ,  Aaron Hertzmann

IPC: G06T13/40 ,  G06N3/08 ,  G06T15/50 ,  G06T15/20

Abstract: In some embodiments, a motion model refinement system receives an input video depicting a human character and an initial motion model describing motions of individual joint points of the human character in a three-dimensional space. The motion model refinement system identifies foot joint points of the human character that are in contact with a ground plane using a trained contact estimation model. The motion model refinement system determines the ground plane based on the foot joint points and the initial motion model and constructs an optimization problem for refining the initial motion model. The optimization problem minimizes the difference between the refined motion model and the initial motion model under a set of plausibility constraints including constraints on the contact foot joint points and a time-dependent inertia tensor-based constraint. The motion model refinement system obtains the refined motion model by solving the optimization problem.

公开(公告)号：US20220139019A1

公开(公告)日：2022-05-05

申请号：US17573890

申请日：2022-01-12

Applicant: Adobe Inc.

Inventor： Jimei Yang ,  Davis Rempe ,  Bryan Russell ,  Aaron Hertzmann

IPC: G06T13/40 ,  G06N3/08 ,  G06T15/50 ,  G06T15/20

Abstract: In some embodiments, a model training system obtains a set of animation models. For each of the animation models, the model training system renders the animation model to generate a sequence of video frames containing a character using a set of rendering parameters and extracts joint points of the character from each frame of the sequence of video frames. The model training system further determines, for each frame of the sequence of video frames, whether a subset of the joint points are in contact with a ground plane in a three-dimensional space and generates contact labels for the subset of the joint points. The model training system trains a contact estimation model using training data containing the joint points extracted from the sequences of video frames and the generated contact labels. The contact estimation model can be used to refine a motion model for a character.