公开(公告)号：US20240005590A1

公开(公告)日：2024-01-04

申请号：US18251995

申请日：2021-01-14

Applicant: GOOGLE LLC

Inventor： Ricardo Martin Brualla ,  Keunhong Park ,  Utkarsh Sinha ,  Sofien Bouaziz ,  Daniel Goldman ,  Jonathan Tilton Barron ,  Steven Maxwell Seitz

IPC: G06T15/20 ,  G06T15/55 ,  G06T15/04

CPC classification number: G06T15/20 ,  G06T15/55 ,  G06T15/04

Abstract: Techniques of image synthesis using a neural radiance field (NeRF) includes generating a deformation model of movement experienced by a subject in a non-rigidly deforming scene. For example, when an image synthesis system uses NeRFs, the system takes as input multiple poses of subjects for training data. In contrast to conventional NeRFs, the technical solution first expresses the positions of the subjects from various perspectives in an observation frame. The technical solution then involves deriving a deformation model, i.e., a mapping between the observation frame and a canonical frame in which the subject's movements are taken into account. This mapping is accomplished using latent deformation codes for each pose that are determined using a multilayer perceptron (MLP). A NeRF is then derived from positions and casted ray directions in the canonical frame using another MLP. New poses for the subject may then be derived using the NeRF.