公开(公告)号：US12243273B2

公开(公告)日：2025-03-04

申请号：US17571285

申请日：2022-01-07

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Zhaoyang Lv ,  Miroslava Slavcheva ,  Tianye Li ,  Michael Zollhoefer ,  Simon Gareth Green ,  Tanner Schmidt ,  Michael Goesele ,  Steven John Lovegrove ,  Christoph Lassner ,  Changil Kim

IPC: G06T7/00

Abstract: In one embodiment, a method includes initializing latent codes respectively associated with times associated with frames in a training video of a scene captured by a camera. For each of the frames, a system (1) generates rendered pixel values for a set of pixels in the frame by querying NeRF using the latent code associated with the frame, a camera viewpoint associated with the frame, and ray directions associated with the set of pixels, and (2) updates the latent code associated with the frame and the NeRF based on comparisons between the rendered pixel values and original pixel values for the set of pixels. Once trained, the system renders output frames for an output video of the scene, wherein each output frame is rendered by querying the updated NeRF using one of the updated latent codes corresponding to a desired time associated with the output frame.

公开(公告)号：US12108141B2

公开(公告)日：2024-10-01

申请号：US16983863

申请日：2020-08-03

Applicant: Meta Platforms Technologies, LLC

Inventor： Andrew Samuel Berkovich ,  Reid Pinkham ,  Tanner Schmidt

IPC: H04N23/617 ,  G06N3/02 ,  H01L27/146 ,  H04N23/60 ,  H04N23/80 ,  H04N25/443

CPC classification number: H04N23/617 ,  G06N3/02 ,  H01L27/14634 ,  H01L27/14636 ,  H04N23/665 ,  H04N23/80 ,  H04N25/443

Abstract: In one example, an apparatus comprises a pixel cell array and a controller formed within a semiconductor package. The pixel cell array is configured to: generate, at a first time and based on first programming signals received from the controller, a first image frame; transmit the first image frame to a host processor; and transmit the first image frame or a second image frame to the controller, the second image frame being generated at the first time and having a different sparsity of pixels from the first image frame. The controller is configured to receive second programming signals from a host processor, the second programming signals being determined by the host processor based on the first image frame; update the first programming signals based on the second programming signals; and control the pixel cell array to generate a subsequent image frame based on the updated first programming signals.