公开(公告)号：US20230419600A1

公开(公告)日：2023-12-28

申请号：US18251743

申请日：2020-11-05

Applicant: GOOGLE LLC

Inventor： Sean Ryan Francesco FANELLO ,  Abhi MEKA ,  Rohit Kumar PANDEY ,  Christian HAENE ,  Sergio Orts ESCOLANO ,  Christoph RHEMANN ,  Paul DEBEVEC ,  Sofien BOUAZIZ ,  Thabo BEELER ,  Ryan OVERBECK ,  Peter BARNUM ,  Daniel ERICKSON ,  Philip DAVIDSON ,  Yinda ZHANG ,  Jonathan TAYLOR ,  Chloe LeGENDRE ,  Shahram IZADI

IPC: G06T15/50 ,  G06T7/55 ,  G06T15/04 ,  G06T15/20 ,  G06T7/60

CPC classification number: G06T15/506 ,  G06T7/55 ,  G06T15/04 ,  G06T15/20 ,  G06T7/60 ,  G06T2207/10048 ,  G06T2207/20084 ,  G06T2207/10152 ,  G06T2207/30196

Abstract: Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.