公开(公告)号：US20220343525A1

公开(公告)日：2022-10-27

申请号：US17640114

申请日：2020-04-27

Applicant: GOOGLE LLC

Inventor： Rahul GARG ,  Neal WADHWA ,  Sean FANELLO ,  Christian HAENE ,  Yinda ZHANG ,  Sergio Orts ESCOLANO ,  Yael Pritch KNAAN ,  Marc LEVOY ,  Shahram IZADI

IPC: G06T7/593 ,  G06T5/00

Abstract: Example implementations relate to joint depth prediction from dual cameras and dual pixels. An example method may involve obtaining a first set of depth information representing a scene from a first source and a second set of depth information representing the scene from a second source. The method may further involve determining, using a neural network, a joint depth map that conveys respective depths for elements in the scene. The neural network may determine the joint depth map based on a combination of the first set of depth information and the second set of depth information. In addition, the method may involve modifying an image representing the scene based on the joint depth map. For example, background portions of the image may be partially blurred based on the joint depth map.

公开(公告)号：US20230419600A1

公开(公告)日：2023-12-28

申请号：US18251743

申请日：2020-11-05

Applicant: GOOGLE LLC

Inventor： Sean Ryan Francesco FANELLO ,  Abhi MEKA ,  Rohit Kumar PANDEY ,  Christian HAENE ,  Sergio Orts ESCOLANO ,  Christoph RHEMANN ,  Paul DEBEVEC ,  Sofien BOUAZIZ ,  Thabo BEELER ,  Ryan OVERBECK ,  Peter BARNUM ,  Daniel ERICKSON ,  Philip DAVIDSON ,  Yinda ZHANG ,  Jonathan TAYLOR ,  Chloe LeGENDRE ,  Shahram IZADI

IPC: G06T15/50 ,  G06T7/55 ,  G06T15/04 ,  G06T15/20 ,  G06T7/60

CPC classification number: G06T15/506 ,  G06T7/55 ,  G06T15/04 ,  G06T15/20 ,  G06T7/60 ,  G06T2207/10048 ,  G06T2207/20084 ,  G06T2207/10152 ,  G06T2207/30196

Abstract: Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.