公开(公告)号：US20210118157A1

公开(公告)日：2021-04-22

申请号：US16643213

申请日：2019-10-21

Applicant: GOOGLE LLC

Inventor： Chloe LeGENDRE ,  Ranjith Kagathi ANANDA ,  Ran TAO ,  Wim MEEUSSEN

IPC: G06T7/37 ,  G06T7/73 ,  G06T11/00

Abstract: Systems, methods, and computer program products are described that include obtaining, at a processor, a first image from an image capture device onboard a computing device, detecting, using the processor and at least one sensor, a device orientation of the computing device associated with capture of the first image, determining, based on the device orientation and a tracking stack associated with the computing device, a rotation angle in which to rotate the first image, rotating the first image to the rotation angle to generate a second image, and generating neural network based estimates associated with the first image and the second image.

公开(公告)号：US20230419600A1

公开(公告)日：2023-12-28

申请号：US18251743

申请日：2020-11-05

Applicant: GOOGLE LLC

Inventor： Sean Ryan Francesco FANELLO ,  Abhi MEKA ,  Rohit Kumar PANDEY ,  Christian HAENE ,  Sergio Orts ESCOLANO ,  Christoph RHEMANN ,  Paul DEBEVEC ,  Sofien BOUAZIZ ,  Thabo BEELER ,  Ryan OVERBECK ,  Peter BARNUM ,  Daniel ERICKSON ,  Philip DAVIDSON ,  Yinda ZHANG ,  Jonathan TAYLOR ,  Chloe LeGENDRE ,  Shahram IZADI

IPC: G06T15/50 ,  G06T7/55 ,  G06T15/04 ,  G06T15/20 ,  G06T7/60

CPC classification number: G06T15/506 ,  G06T7/55 ,  G06T15/04 ,  G06T15/20 ,  G06T7/60 ,  G06T2207/10048 ,  G06T2207/20084 ,  G06T2207/10152 ,  G06T2207/30196

Abstract: Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.