公开(公告)号：US20190088023A1

公开(公告)日：2019-03-21

申请号：US16113404

申请日：2018-08-27

Applicant: Google LLC

Inventor： Manfred ERNST ,  Daniel ERICKSON ,  Harrison McKenzie CHAPTER

IPC: G06T19/00 ,  H04N13/366 ,  H04N13/117 ,  G02B27/01 ,  G06T7/73 ,  G06T15/20 ,  G06T3/40 ,  H04N13/344

Abstract: An HMD device identifies a pose of the device and identifies a subset of a plurality of camera viewpoints of a light-field based on the pose. The HMD device interpolates image data of the light-field based on the pose and the subset of the plurality of camera viewpoints to generate an interpolated view; and displays at the HMD device an image based on the interpolated view. By interpolating based on the subset of camera viewpoints, the HMD device can reduce processing overhead and improve the user experience.

公开(公告)号：US20230419600A1

公开(公告)日：2023-12-28

申请号：US18251743

申请日：2020-11-05

Applicant: GOOGLE LLC

Inventor： Sean Ryan Francesco FANELLO ,  Abhi MEKA ,  Rohit Kumar PANDEY ,  Christian HAENE ,  Sergio Orts ESCOLANO ,  Christoph RHEMANN ,  Paul DEBEVEC ,  Sofien BOUAZIZ ,  Thabo BEELER ,  Ryan OVERBECK ,  Peter BARNUM ,  Daniel ERICKSON ,  Philip DAVIDSON ,  Yinda ZHANG ,  Jonathan TAYLOR ,  Chloe LeGENDRE ,  Shahram IZADI

IPC: G06T15/50 ,  G06T7/55 ,  G06T15/04 ,  G06T15/20 ,  G06T7/60

CPC classification number: G06T15/506 ,  G06T7/55 ,  G06T15/04 ,  G06T15/20 ,  G06T7/60 ,  G06T2207/10048 ,  G06T2207/20084 ,  G06T2207/10152 ,  G06T2207/30196

Abstract: Example embodiments relate to techniques for volumetric performance capture with neural rendering. A technique may involve initially obtaining images that depict a subject from multiple viewpoints and under various lighting conditions using a light stage and depth data corresponding to the subject using infrared cameras. A neural network may extract features of the subject from the images based on the depth data and map the features into a texture space (e.g., the UV texture space). A neural renderer can be used to generate an output image depicting the subject from a target view such that illumination of the subject in the output image aligns with the target view. The neural render may resample the features of the subject from the texture space to an image space to generate the output image.