公开(公告)号：US20240013462A1

公开(公告)日：2024-01-11

申请号：US17859615

申请日：2022-07-07

Applicant: Nvidia Corporation

Inventor： Yeongho Seol ,  Simon Yuen ,  Dmitry Aleksandrovich Korobchenko ,  Mingquan Zhou ,  Ronan Browne ,  Wonmin Byeon

IPC: G06T13/20 ,  G06T13/40 ,  G06T17/20 ,  G10L25/63 ,  G10L15/16

CPC classification number: G06T13/205 ,  G06T13/40 ,  G06T17/20 ,  G10L25/63 ,  G10L15/16

Abstract: A deep neural network can be trained to output motion or deformation information for a character that is representative of the character uttering speech contained in audio input, which is accurate for an emotional state of the character. The character can have different facial components or regions (e.g., head, skin, eyes, tongue) modeled separately, such that the network can output motion or deformation information for each of these different facial components. During training, the network can be provided with emotion and/or style vectors that indicate information to be used in generating realistic animation for input speech, as may relate to one or more emotions to be exhibited by the character, a relative weighting of those emotions, and any style or adjustments to be made to how the character expresses that emotional state. The network output can be provided to a renderer to generate audio-driven facial animation that is emotion-accurate.