公开(公告)号：US20250116864A1

公开(公告)日：2025-04-10

申请号：US18484215

申请日：2023-10-10

Applicant: Meta Platforms Technologies, LLC

Inventor： Steven Paul Lansel

IPC: G02B27/01 ,  G06F3/01 ,  G06T19/00

Abstract: A method implemented by a computing device includes rendering on a first display and a second display of the computing device an extended reality (XR) environment, and determining a context of the XR environment with respect to a user. Determining the context includes determining characteristics associated with each eye of the user with respect to virtual content displayed within the XR environment. The method includes determining, based on the characteristics, a mapping of a first set of coordinates of the virtual content as displayed on the first display and a second set of coordinates of the virtual content as displayed on the second display, generating, based on the mapping of the first set of coordinates and the second set of coordinates, composite virtual content to be rendered on the first display and the second display; and re-rendering on the first display and the second display the composite virtual content.

公开(公告)号：US20230128288A1

公开(公告)日：2023-04-27

申请号：US18048602

申请日：2022-10-21

Applicant: Meta Platforms Technologies, LLC

Inventor： Steven Paul Lansel ,  Guodong Rong ,  Jian Zhang

IPC: G06T15/50 ,  G06T19/00 ,  G06T15/20 ,  G06F3/01 ,  G06T3/00 ,  G06T5/00 ,  G06T19/20 ,  G06T1/60

Abstract: In one embodiment, a method may obtain, from an application, (a) an image and (b) a layer frame having a first pose in front of the image. The method may generate, for a first viewpoint associated with a first time, a first display frame by separately rendering the image and the layer frame having the first pose into a display buffer. The method may display the first display frame at the first time. The method may determine an extrapolated pose for the layer frame based on the first pose of the layer frame and a second pose of a previously submitted layer frame. The method may generate, for a second viewpoint associated with a second time, a second display frame by separately rendering the image and the layer frame having the extrapolated pose into the display buffer. The method may display the second display frame at the second time.

公开(公告)号：US12271995B2

公开(公告)日：2025-04-08

申请号：US18048602

申请日：2022-10-21

Applicant: Meta Platforms Technologies, LLC

Inventor： Steven Paul Lansel ,  Guodong Rong ,  Jian Zhang

IPC: G06T15/50 ,  G06F3/01 ,  G06T1/60 ,  G06T3/18 ,  G06T5/80 ,  G06T15/20 ,  G06T19/00 ,  G06T19/20

Abstract: In one embodiment, a method may obtain, from an application, (a) an image and (b) a layer frame having a first pose in front of the image. The method may generate, for a first viewpoint associated with a first time, a first display frame by separately rendering the image and the layer frame having the first pose into a display buffer. The method may display the first display frame at the first time. The method may determine an extrapolated pose for the layer frame based on the first pose of the layer frame and a second pose of a previously submitted layer frame. The method may generate, for a second viewpoint associated with a second time, a second display frame by separately rendering the image and the layer frame having the extrapolated pose into the display buffer. The method may display the second display frame at the second time.

公开(公告)号：US11734808B2

公开(公告)日：2023-08-22

申请号：US17848215

申请日：2022-06-23

Applicant: Meta Platforms Technologies, LLC

Inventor： Rohit Rao Padebettu ,  Steven Paul Lansel ,  Todd Douglas Keeler

IPC: G06T5/50 ,  G06T3/00 ,  G06T5/00

CPC classification number: G06T5/50 ,  G06T3/0093 ,  G06T5/006 ,  G06T2207/10016 ,  G06T2207/20004 ,  G06T2207/20221

Abstract: In one embodiment, a method includes receiving a source image and its associated parameters from each of multiple image sources, associating each of the source images with a layer in a range of layers based on the parameters associated with the source images, the range of layers specifying a composition layering order of the source images, generating a corresponding customized distortion mesh for each particular source image in the source images based on the parameters associated with the particular source image and at least a portion of the parameters associated with each of the source images that is associated with any layer preceding a layer associated with the particular source image, modifying each of the source images using the corresponding customized distortion mesh, generate a composite image using the modified source images, and display the composite image as a frame in a video.

公开(公告)号：US20250168320A1

公开(公告)日：2025-05-22

申请号：US18938148

申请日：2024-11-05

Applicant: Meta Platforms Technologies, LLC

Inventor： Rohit Rao Padebettu ,  Nghiep Chinh Luu ,  Irad Ratmansky ,  Steven Paul Lansel

IPC: H04N13/366 ,  H04N13/139 ,  H04N13/156 ,  H04N13/398

Abstract: An XR device includes a display device, a lens, and programs. The programs are stored in memory and configured to be executed by processors. Moreover, the programs include instructions for determining an HPPD metric for the XR device based on characteristics of the display device and the lens. The programs also include instructions for receiving data regarding a size and a location of visual XR content from an XR application running on the XR device. Additionally, the programs include instructions for determining an RPPD metric for the visual XR content based on the data. Further, the programs include instructions for determining a recommended resolution for rendering the visual XR content based on the HPPD metric and the RPPD metric. Moreover, the programs include instructions for transmitting the recommended resolution to the XR application, where the XR application is configured to render the visual XR content using the recommended resolution.

公开(公告)号：US12069230B2

公开(公告)日：2024-08-20

申请号：US17132228

申请日：2020-12-23

Applicant: Meta Platforms Technologies, LLC

Inventor： Behnam Bastani ,  Steven Paul Lansel ,  Todd Douglas Keeler

IPC: H04N13/332 ,  G06T7/269

CPC classification number: H04N13/332 ,  G06T7/269 ,  G06T2207/10016

Abstract: In one embodiment, a method includes accessing a first image corresponding to a first frame of a video stream, where the first image has complete pixel information, rendering a provisional image corresponding to a second frame of the video stream subsequent to the first frame, where the provisional image has a first area with complete pixel information and a second area with incomplete pixel information, generating a predicted image corresponding to the second frame by re-projecting at least an area of the first image according to one or more warping parameters, and generating a second image corresponding to the second frame by compositing the rendered provisional image and the predicted image.

公开(公告)号：US20240249478A1

公开(公告)日：2024-07-25

申请号：US18398678

申请日：2023-12-28

Applicant: Meta Platforms Technologies, LLC

Inventor： Sebastian Sztuk ,  Ilya Brailovskiy ,  Steven Paul Lansel ,  Grant Kaijuin Yang

IPC: G06T19/00 ,  G06T3/4053

CPC classification number: G06T19/006 ,  G06T3/4053

Abstract: A method implemented by a computing device includes rendering on displays of a computing device an extended reality (XR) environment, and determining a context of the XR environment with respect to a user. Determining the context includes determining characteristics associated with an eye of the user with respect to content displayed. The method includes generating, based on the characteristics associated with the eye, a foveated map including a plurality of foveal regions. The plurality of foveal regions includes a plurality of zones each corresponding to a low-resolution area of the content for the respective zone. The method includes inputting one or more of the plurality of zones into a machine-learning model trained to generate a super-resolution reconstruction of the foveated map based on regions of interest identified within the one or more of the plurality of zones, and outputting, by the machine-learning model, the super-resolution reconstruction of the foveated map.

公开(公告)号：US12039695B1

公开(公告)日：2024-07-16

申请号：US17666332

申请日：2022-02-07

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Haomiao Jiang ,  Todd Douglas Keeler ,  Grant Kaijuin Yang ,  Rohit Rao Padebettu ,  Steven Paul Lansel ,  Behnam Bastani

IPC: G06T5/00 ,  G06T3/4046 ,  G06T3/4053 ,  G06T15/00

CPC classification number: G06T3/4046 ,  G06T3/4053 ,  G06T15/005 ,  G06T2200/24

Abstract: In particular embodiments, the disclosure provides a method comprising: rendering, on a graphics processing unit (GPU), a low-resolution image associated with a scene, the low-resolution image having a resolution that is lower than a target resolution; transmitting a version of the low-resolution image to a neural accelerator; processing, on the neural accelerator, the version of the low-resolution image using a trained machine-learning model, thereby outputting a plurality of control parameters; transmitting the control parameters from the neural accelerator to the GPU; processing, on the GPU, the low-resolution image and the control parameters to construct a high-resolution image having the target resolution, wherein the GPU is programmed to determine a plurality of pixel weights for performing an interpolation using the control parameters; and outputting the high-resolution image.

公开(公告)号：US20230134355A1

公开(公告)日：2023-05-04

申请号：US17974826

申请日：2022-10-27

Applicant: Meta Platforms Technologies, LLC

Inventor： Steven Paul Lansel ,  Todd Douglas Keeler ,  Rohit Rao Padebettu ,  Alexander Michael Louie ,  Michal Hlavac ,  Wai Leong Chak ,  Yeliz Karadayi

IPC: G06T11/00 ,  G06T3/00 ,  G06T1/60 ,  G06F9/54

Abstract: In one embodiment, a method includes accessing a first image corresponding to a first frame of a video stream, rendering a first area of a second image corresponding to a second frame of the video stream, generating a second area of the second image corresponding to the second frame of the video stream by re-projecting the second area of the first image according to one or more warping parameters, and constructing the second image corresponding to the second frame by compositing the rendered first area and the generated second area of the second image.
In another embodiment, a method includes an operating system receiving a set of data associated with an object from a first application, storing the set of data on the operating system, receiving a command to share the object with a second application, and allowing the second application to access the portion of the data associated with the object that it needs.

公开(公告)号：US20230245260A1

公开(公告)日：2023-08-03

申请号：US18162297

申请日：2023-01-31

Applicant: Meta Platforms Technologies, LLC

Inventor： Weihua Gao ,  Todd Douglas Keeler ,  Steven Paul Lansel ,  Jian Zhang ,  Tianxin Ning

IPC: G06T1/20 ,  G06T5/00 ,  G06T7/11 ,  G06F3/01

CPC classification number: G06T1/20 ,  G06T5/002 ,  G06T7/11 ,  G06F3/013

Abstract: In one embodiment, a method includes by a computing system, rendering an image using a tile-based graphics processing unit. Determining a gaze location of a user wearing a head-mounted device and using the gaze location to select, from the multiple tiles, central tiles in which the user's gaze location is located, periphery tiles outside of the central tiles, and border tiles located between the central tiles and the periphery tiles. Instructing the GPU to render (a) the central tiles in a first pixel-density, (b) the periphery tiles in a second pixel-density, and (c) the border tiles in the first pixel-density and in the second pixel-density and then blending the border tiles rendered in the first pixel-density and the border tiles rendered in the second pixel-density to create blended border tiles. Then, outputting the central tiles, the periphery tiles, and the blended border tiles using a display of the head-mounted device.