公开(公告)号：US11734888B2

公开(公告)日：2023-08-22

申请号：US17396534

申请日：2021-08-06

Applicant: Meta Platforms Technologies, LLC

Inventor： Chen Cao ,  Vasu Agrawal ,  Fernando De la Torre ,  Lele Chen ,  Jason Saragih ,  Tomas Simon Kreuz ,  Yaser Sheikh

IPC: G06T17/20 ,  G06T7/586 ,  G06T7/73 ,  G06V40/16 ,  G06T13/40 ,  G06T15/04 ,  G06N3/045

CPC classification number: G06T17/20 ,  G06N3/045 ,  G06T7/586 ,  G06T7/73 ,  G06T13/40 ,  G06T15/04 ,  G06V40/165 ,  G06V40/176

Abstract: A method for providing real-time three-dimensional facial animation from video is provided. The method includes collecting images of a subject, and forming a three-dimensional mesh for the subject based on a facial expression factor and a head pose of the subject extracted from the images of the subject. The method also includes forming a texture transformation based on an illumination parameter associated with an illumination configuration for the images from the subject, forming a three-dimensional model for the subject based on the three-dimensional mesh and the texture transformation, determining a loss factor based on selected points in a test image from the subject and a rendition of the test image by the three-dimensional model, and updating the three-dimensional model according to the loss factor. A system and a non-transitory, computer-readable medium storing instructions to perform the above method are also provided.

公开(公告)号：US20220358719A1

公开(公告)日：2022-11-10

申请号：US17396534

申请日：2021-08-06

Applicant: Meta Platforms Technologies, LLC

Inventor： Chen Cao ,  Vasu Agrawal ,  Fernando De la Torre ,  Lele Chen ,  Jason Saragih ,  Tomas Simon Kreuz ,  Yaser Sheikh

IPC: G06T17/20 ,  G06T15/04 ,  G06T13/40 ,  G06T7/73 ,  G06N3/04 ,  G06K9/00 ,  G06T7/586

Abstract: A method for providing real-time three-dimensional facial animation from video is provided. The method includes collecting images of a subject, and forming a three-dimensional mesh for the subject based on a facial expression factor and a head pose of the subject extracted from the images of the subject. The method also includes forming a texture transformation based on an illumination parameter associated with an illumination configuration for the images from the subject, forming a three-dimensional model for the subject based on the three-dimensional mesh and the texture transformation, determining a loss factor based on selected points in a test image from the subject and a rendition of the test image by the three-dimensional model, and updating the three-dimensional model according to the loss factor. A system and a non-transitory, computer-readable medium storing instructions to perform the above method are also provided.