公开(公告)号：US20230177756A1

公开(公告)日：2023-06-08

申请号：US18075346

申请日：2022-12-05

Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.

Inventor： Zhe PENG ,  Yuqiang LIU ,  Fanyu GENG

IPC: G06T13/40 ,  G06T7/20 ,  G10L25/57 ,  G10L15/06 ,  G10L15/02 ,  G10L15/16

CPC classification number: G06T13/40 ,  G06T7/20 ,  G10L25/57 ,  G10L15/063 ,  G10L15/02 ,  G10L15/16 ,  G06T2207/30201 ,  G06T2207/20084 ,  G06T2207/20081

Abstract: A method of generating a 3D video, a method of training a neural network model, an electronic device, and a storage medium, which relate to a field of image processing, and in particular to technical fields of computer vision, augmented/virtual reality and deep learning. The method includes: determining, based on an input speech feature, a principal component analysis (PCA) coefficient by using a first network, wherein the PCA coefficient is used to generate the 3D video; correcting the PCA coefficient by using a second network; generating a lip movement information based on the corrected PCA coefficient and a PCA parameter for a neural network model, wherein the neural network model includes the first network and the second network; and applying the lip movement information to a pre-constructed 3D basic avatar model to obtain a 3D video with a lip movement effect.