公开(公告)号：US20240249478A1

公开(公告)日：2024-07-25

申请号：US18398678

申请日：2023-12-28

Applicant: Meta Platforms Technologies, LLC

Inventor： Sebastian Sztuk ,  Ilya Brailovskiy ,  Steven Paul Lansel ,  Grant Kaijuin Yang

IPC: G06T19/00 ,  G06T3/4053

CPC classification number: G06T19/006 ,  G06T3/4053

Abstract: A method implemented by a computing device includes rendering on displays of a computing device an extended reality (XR) environment, and determining a context of the XR environment with respect to a user. Determining the context includes determining characteristics associated with an eye of the user with respect to content displayed. The method includes generating, based on the characteristics associated with the eye, a foveated map including a plurality of foveal regions. The plurality of foveal regions includes a plurality of zones each corresponding to a low-resolution area of the content for the respective zone. The method includes inputting one or more of the plurality of zones into a machine-learning model trained to generate a super-resolution reconstruction of the foveated map based on regions of interest identified within the one or more of the plurality of zones, and outputting, by the machine-learning model, the super-resolution reconstruction of the foveated map.

公开(公告)号：US12039695B1

公开(公告)日：2024-07-16

申请号：US17666332

申请日：2022-02-07

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Haomiao Jiang ,  Todd Douglas Keeler ,  Grant Kaijuin Yang ,  Rohit Rao Padebettu ,  Steven Paul Lansel ,  Behnam Bastani

IPC: G06T5/00 ,  G06T3/4046 ,  G06T3/4053 ,  G06T15/00

CPC classification number: G06T3/4046 ,  G06T3/4053 ,  G06T15/005 ,  G06T2200/24

Abstract: In particular embodiments, the disclosure provides a method comprising: rendering, on a graphics processing unit (GPU), a low-resolution image associated with a scene, the low-resolution image having a resolution that is lower than a target resolution; transmitting a version of the low-resolution image to a neural accelerator; processing, on the neural accelerator, the version of the low-resolution image using a trained machine-learning model, thereby outputting a plurality of control parameters; transmitting the control parameters from the neural accelerator to the GPU; processing, on the GPU, the low-resolution image and the control parameters to construct a high-resolution image having the target resolution, wherein the GPU is programmed to determine a plurality of pixel weights for performing an interpolation using the control parameters; and outputting the high-resolution image.