公开(公告)号：US12069230B2

公开(公告)日：2024-08-20

申请号：US17132228

申请日：2020-12-23

Applicant: Meta Platforms Technologies, LLC

Inventor： Behnam Bastani ,  Steven Paul Lansel ,  Todd Douglas Keeler

IPC: H04N13/332 ,  G06T7/269

CPC classification number: H04N13/332 ,  G06T7/269 ,  G06T2207/10016

Abstract: In one embodiment, a method includes accessing a first image corresponding to a first frame of a video stream, where the first image has complete pixel information, rendering a provisional image corresponding to a second frame of the video stream subsequent to the first frame, where the provisional image has a first area with complete pixel information and a second area with incomplete pixel information, generating a predicted image corresponding to the second frame by re-projecting at least an area of the first image according to one or more warping parameters, and generating a second image corresponding to the second frame by compositing the rendered provisional image and the predicted image.

公开(公告)号：US11734808B2

公开(公告)日：2023-08-22

申请号：US17848215

申请日：2022-06-23

Applicant: Meta Platforms Technologies, LLC

Inventor： Rohit Rao Padebettu ,  Steven Paul Lansel ,  Todd Douglas Keeler

IPC: G06T5/50 ,  G06T3/00 ,  G06T5/00

CPC classification number: G06T5/50 ,  G06T3/0093 ,  G06T5/006 ,  G06T2207/10016 ,  G06T2207/20004 ,  G06T2207/20221

Abstract: In one embodiment, a method includes receiving a source image and its associated parameters from each of multiple image sources, associating each of the source images with a layer in a range of layers based on the parameters associated with the source images, the range of layers specifying a composition layering order of the source images, generating a corresponding customized distortion mesh for each particular source image in the source images based on the parameters associated with the particular source image and at least a portion of the parameters associated with each of the source images that is associated with any layer preceding a layer associated with the particular source image, modifying each of the source images using the corresponding customized distortion mesh, generate a composite image using the modified source images, and display the composite image as a frame in a video.

公开(公告)号：US12039695B1

公开(公告)日：2024-07-16

申请号：US17666332

申请日：2022-02-07

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Haomiao Jiang ,  Todd Douglas Keeler ,  Grant Kaijuin Yang ,  Rohit Rao Padebettu ,  Steven Paul Lansel ,  Behnam Bastani

IPC: G06T5/00 ,  G06T3/4046 ,  G06T3/4053 ,  G06T15/00

CPC classification number: G06T3/4046 ,  G06T3/4053 ,  G06T15/005 ,  G06T2200/24

Abstract: In particular embodiments, the disclosure provides a method comprising: rendering, on a graphics processing unit (GPU), a low-resolution image associated with a scene, the low-resolution image having a resolution that is lower than a target resolution; transmitting a version of the low-resolution image to a neural accelerator; processing, on the neural accelerator, the version of the low-resolution image using a trained machine-learning model, thereby outputting a plurality of control parameters; transmitting the control parameters from the neural accelerator to the GPU; processing, on the GPU, the low-resolution image and the control parameters to construct a high-resolution image having the target resolution, wherein the GPU is programmed to determine a plurality of pixel weights for performing an interpolation using the control parameters; and outputting the high-resolution image.

公开(公告)号：US20230134355A1

公开(公告)日：2023-05-04

申请号：US17974826

申请日：2022-10-27

Applicant: Meta Platforms Technologies, LLC

Inventor： Steven Paul Lansel ,  Todd Douglas Keeler ,  Rohit Rao Padebettu ,  Alexander Michael Louie ,  Michal Hlavac ,  Wai Leong Chak ,  Yeliz Karadayi

IPC: G06T11/00 ,  G06T3/00 ,  G06T1/60 ,  G06F9/54

Abstract: In one embodiment, a method includes accessing a first image corresponding to a first frame of a video stream, rendering a first area of a second image corresponding to a second frame of the video stream, generating a second area of the second image corresponding to the second frame of the video stream by re-projecting the second area of the first image according to one or more warping parameters, and constructing the second image corresponding to the second frame by compositing the rendered first area and the generated second area of the second image.
In another embodiment, a method includes an operating system receiving a set of data associated with an object from a first application, storing the set of data on the operating system, receiving a command to share the object with a second application, and allowing the second application to access the portion of the data associated with the object that it needs.

公开(公告)号：US20230245260A1

公开(公告)日：2023-08-03

申请号：US18162297

申请日：2023-01-31

Applicant: Meta Platforms Technologies, LLC

Inventor： Weihua Gao ,  Todd Douglas Keeler ,  Steven Paul Lansel ,  Jian Zhang ,  Tianxin Ning

IPC: G06T1/20 ,  G06T5/00 ,  G06T7/11 ,  G06F3/01

CPC classification number: G06T1/20 ,  G06T5/002 ,  G06T7/11 ,  G06F3/013

Abstract: In one embodiment, a method includes by a computing system, rendering an image using a tile-based graphics processing unit. Determining a gaze location of a user wearing a head-mounted device and using the gaze location to select, from the multiple tiles, central tiles in which the user's gaze location is located, periphery tiles outside of the central tiles, and border tiles located between the central tiles and the periphery tiles. Instructing the GPU to render (a) the central tiles in a first pixel-density, (b) the periphery tiles in a second pixel-density, and (c) the border tiles in the first pixel-density and in the second pixel-density and then blending the border tiles rendered in the first pixel-density and the border tiles rendered in the second pixel-density to create blended border tiles. Then, outputting the central tiles, the periphery tiles, and the blended border tiles using a display of the head-mounted device.

公开(公告)号：US20230136662A1

公开(公告)日：2023-05-04

申请号：US18047924

申请日：2022-10-19

Applicant: Meta Platforms Technologies, LLC

Inventor： Todd Douglas Keeler ,  Steven Paul Lansel

IPC: G06T3/00 ,  G06T3/40 ,  G06T7/20 ,  G06T7/70

Abstract: In one embodiment, a method includes obtaining a first frame rendered for a first head pose and a second frame rendered for a second head pose, generating first motion vectors based on a first comparison between the first frame and the second frame, determining a first positional displacement vector based on the first head pose and the second head pose, determining a second positional displacement vector based on the second head pose and a subsequent head pose, generating a positional extrapolation for the subsequent head pose by projecting the second positional displacement vector onto the first positional displacement vector, generating a scaling factor based on the positional extrapolation, updating the second frame based on the scaling factor and the first motion vectors, and rendering a subsequent frame for the subsequent head pose based on the updated second frame.