公开(公告)号：US12131416B2

公开(公告)日：2024-10-29

申请号：US17556367

申请日：2021-12-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Stephen Anthony Lombardi ,  Jason Saragih ,  Tomas Simon Kreuz ,  Shunsuke Saito ,  Michael Zollhoefer ,  Amit Raj ,  James Henry Hays

IPC: G06T13/40 ,  G06T7/00 ,  G06T7/73

CPC classification number: G06T13/40 ,  G06T7/73 ,  G06T7/97 ,  G06T2207/30201

Abstract: A method of forming a pixel-aligned volumetric avatar includes receiving multiple two-dimensional images having at least two or more fields of view of a subject. The method also includes extracting multiple image features from the two-dimensional images using a set of learnable weights, projecting the image features along a direction between a three-dimensional model of the subject and a selected observation point for a viewer, and providing, to the viewer, an image of the three-dimensional model of the subject. A system and a non-transitory, computer readable medium storing instructions to perform the above method, are also provided.

公开(公告)号：US20240406371A1

公开(公告)日：2024-12-05

申请号：US18800312

申请日：2024-08-12

Applicant: Meta Platforms Technologies, LLC

Inventor： Nathan Matsuda ,  Brian Wheelwright ,  Joel Hegland ,  Stephen Anthony Lombardi ,  Jason Saragih ,  Tomas Simon Kreuz ,  Shunsuke Saito ,  Michael Zollhoefer ,  Amit Raj ,  James Henry Hays

IPC: H04N13/344 ,  G02B30/10 ,  G06T7/80 ,  G06T15/00 ,  H04N5/33 ,  H04N13/117 ,  H04N13/239 ,  H04N13/383

Abstract: A device for providing a reverse pass-through view of a user of a headset display to an onlooker includes an eyepiece comprising an optical surface configured to provide an image to a user on a first side of the optical surface. The device also includes a first camera configured to collect an image of a portion of a face of the user reflected from the optical surface in a first field of view, a display adjacent to the optical surface and configured to project forward an image of the face of the user, and a screen configured to receive light from the display and provide the image of the face of the user to an onlooker.

公开(公告)号：US12095975B2

公开(公告)日：2024-09-17

申请号：US17555037

申请日：2021-12-17

Applicant: Meta Platforms Technologies, LLC

Inventor： Nathan Matsuda ,  Brian Wheelwright ,  Joel Hegland ,  Stephen Anthony Lombardi ,  Jason Saragih ,  Tomas Simon Kreuz ,  Shunsuke Saito ,  Michael Zollhoefer ,  Amit Raj ,  James Henry Hays

IPC: H04N13/344 ,  G02B30/10 ,  G06T7/80 ,  G06T15/00 ,  H04N5/33 ,  H04N13/117 ,  H04N13/239 ,  H04N13/383

CPC classification number: H04N13/344 ,  G02B30/10 ,  G06T7/80 ,  G06T15/00 ,  H04N5/33 ,  H04N13/117 ,  H04N13/239 ,  H04N13/383 ,  G06T2207/20081

Abstract: A device for providing a reverse pass-through view of a user of a headset display to an onlooker includes an eyepiece comprising an optical surface configured to provide an image to a user on a first side of the optical surface. The device also includes a first camera configured to collect an image of a portion of a face of the user reflected from the optical surface in a first field of view, a display adjacent to the optical surface and configured to project forward an image of the face of the user, and a screen configured to receive light from the display and provide the image of the face of the user to an onlooker.

公开(公告)号：US11651540B2

公开(公告)日：2023-05-16

申请号：US17228364

申请日：2021-04-12

Applicant: Meta Platforms Technologies, LLC

Inventor： Minh Phuoc Vo ,  Christoph Lassner ,  Carsten Sebastian Stoll ,  Amit Raj

IPC: G06T7/70 ,  G06T7/90 ,  G06T13/40 ,  G06T15/04 ,  G06T15/50 ,  G06N3/08

CPC classification number: G06T13/40 ,  G06N3/08 ,  G06T7/70 ,  G06T7/90 ,  G06T15/04 ,  G06T15/503 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30196

Abstract: In one embodiment, a method includes adjusting parameters of a three-dimensional geometry corresponding to a first person to make the three-dimensional geometry represent a desired pose for the first person, accessing a neural texture encoding an appearance of the first person, generating a first rendered neural texture based on a mapping between (1) a portion of the three-dimensional geometry that is visible from a viewing direction and (2) the neural texture, generating a second rendered neural texture by processing the first rendered neural texture using a first neural network, determining normal information associated with the portion of the three-dimensional geometry that is visible from the viewing direction, and generating a rendered image for the first person in the desired pose by processing the second rendered neural texture and the normal information using a second neural network.