公开(公告)号：US20240193412A1

公开(公告)日：2024-06-13

申请号：US18063843

申请日：2022-12-09

Applicant: Lemon Inc.

Inventor： Song Bai ,  Zhongcong Xu ,  Jiashi Feng ,  Jun Hao Liew ,  Wenqing Zhang

IPC: G06N3/08

CPC classification number: G06N3/08 ,  G06T2207/20081

Abstract: Generating a multi-dimensional video using a multi-dimensional video generative model for, including, but not limited to, at least one of static portrait animation, video reconstruction, or motion editing. The method including providing data into the multi-dimensionally aware generator of the multi-dimensional video generative model, and generating the multi-dimensional video from the data by the multi-dimensionally aware generator. The generating of the multi-dimensional video includes inverting the data into a latent space of the multi-dimensionally aware generator, synthesizing content of the multi-dimensional video using an appearance component of the multi-dimensionally aware generator and corresponding camera pose and formulating an intermediate appearance code, developing a synthesis layer for encoding a motion component of the multi-dimensionally aware generator at a plurality of timesteps and formulating an intermediate motion code, introducing temporal dynamics into the intermediate appearance code and the intermediate motion code, and generating multi-dimensionally aware spatio-temporal representations of the data.