公开(公告)号：US20250157130A1

公开(公告)日：2025-05-15

申请号：US18418130

申请日：2024-01-19

Applicant: Meta Platforms Technologies, LLC

Inventor： Andrey Tovchigrechko ,  Alexander Sorkine Hornung ,  Sebastian Sztuk ,  Remo Ziegler ,  Oskar Linde ,  Ricardo da Silveira Cabral ,  Alex Locher ,  Anirud Thyagharajan ,  Artem Filatov ,  Fabian Langguth ,  Ngoc Minh Dang ,  Salah Eddine Nouri ,  Paul Timothy Furgale

IPC: G06T15/10 ,  G06T5/77 ,  G06T7/194 ,  G06T7/50 ,  G06T19/00

Abstract: A device may access an input image of a real-world scene captured by a camera from a camera viewpoint. The device may render, from the camera viewpoint, an inpainting image of a background of the real-world scene based on a 3D reconstruction model of the background of the real-world scene, wherein the 3D reconstruction model is generated using previously-captured images and previously-generated depth estimates. The device may generate a depth estimate of the real-world scene and identify, based on the depth estimate, a first set of pixel locations and a second set of pixel locations in a passthrough image to be rendered. The device may render, from a viewpoint of an eye of a user, the passthrough image based on the input image, the inpainting image, and the depth estimate. First pixel values for the first set of pixel locations in the passthrough image are sampled from the input image, and second pixel values for the second set of pixel locations in the passthrough image are sampled from the inpainting image.