公开(公告)号：US20250028389A1

公开(公告)日：2025-01-23

申请号：US18909167

申请日：2024-10-08

Applicant: Meta Platforms Technologies, LLC

Inventor： Qing Chao ,  Robert Dale Cavin ,  Guillaume Lestoquoy ,  Mohammadhossein Daraeihajitooei ,  Xinqiao Liu ,  Raffaele Capoccia ,  Sascha Hallstein ,  Ziyun Li

IPC: G06F3/01 ,  G06T7/73 ,  G06V10/25 ,  G06V10/60 ,  G06V40/18 ,  H04N5/33

Abstract: Imaging signals are generated in response to image light. Event signals are generated in response to receive the imaging signals from imaging pixels. A Region of Interest (ROI) of the imaging pixels is identified from a spatial concentration of event signals in the ROI of imaging pixels within a time period. An ROI portion of the imaging pixels in the ROI are driven to capture an ROI image frame.

公开(公告)号：US20230260268A1

公开(公告)日：2023-08-17

申请号：US17707149

申请日：2022-03-29

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Syed Shakib Sarwar ,  Manan Suri ,  Vivek Kamalkant Parmar ,  Ziyun Li ,  Barbara De Salvo ,  Hsien-Hsin Sean Lee

IPC: G06V10/82 ,  G06T19/00 ,  G06F3/01 ,  G06T7/80 ,  G06V40/10 ,  G06V10/764 ,  G06V40/16

CPC classification number: G06V10/82 ,  G06T19/006 ,  G06F3/013 ,  G06T7/80 ,  G06V40/11 ,  G06V10/764 ,  G06V40/161 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A console and headset system locally trains machine learning models to perform customized online learning tasks. To customize the online learning models for specific users of the system without using outside resources, the system trains the models to compare a target frame to stored calibration frames, rather than directly inferring information about a target frame. During deployment, an embedding is generated for the target frame. A sample embedding that is closest to the target embedding is selected from a group of embeddings of calibration frames. The information about the selected embedding and target embedding and ground truths for the calibration frame are provided as inputs to one of the trained models. The model predicts a difference between the target frame and the calibration frame, which can be used to determine information about the target frame.

公开(公告)号：US20230032925A1

公开(公告)日：2023-02-02

申请号：US17729951

申请日：2022-04-26

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Ziyun Li ,  Barbara De Salvo ,  Xinqiao Liu ,  Lyle David Bainbridge ,  Andrew Samuel Berkovich ,  Syed Shakib Sarwar ,  Song Chen ,  Tsung-Hsun Tsai

IPC: H04N5/232 ,  G06V10/147 ,  G06V10/94 ,  G06V10/10 ,  G06T7/20 ,  G06V10/70

Abstract: The disclosed system may include a first layer that includes multiple digital pixel sensors configured to detect light. The system may also include a second layer that includes various image processing components configured to process the light detected by the digital pixel sensors. Still further, the system may include a third layer that includes machine learning (ML) hardware processing components. The image processing components of the second layer may be communicatively connected to the ML hardware processing components of the third layer via multiple micro through-silicon vias (uTSVs). Various other methods of manufacturing, apparatuses, and computer-readable media are also disclosed.

公开(公告)号：US11823498B1

公开(公告)日：2023-11-21

申请号：US17384209

申请日：2021-07-23

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Chengde Wan ,  Randi Cabezas ,  Xinqiao Liu ,  Ziyun Li

IPC: G06V40/20 ,  G06F3/01 ,  G06T7/70

CPC classification number: G06V40/28 ,  G06F3/017 ,  G06T7/70

Abstract: The disclosed computer-implemented method may include (1) receiving a present frame of a video stream, the present frame comprising a present depiction of a multi-segment articulated body system, (2) identifying a previous frame of the video stream that comprises a previous depiction of the multi-segment articulated body system, (3) analyzing the present frame and the previous frame to determine whether the multi-segment articulated body system remained substantially rigid between the previous frame and the present frame, and (4) estimating a pose of the multi-segment articulated body system in the present frame using a first pose estimation computation that treats the multi-segment articulated body system as rigid and that is selected in contrast to a second pose estimation computation based on determining that the multi-segment articulated body system remained substantially rigid between the previous frame and the present frame. Various other methods, systems, and computer-readable media are also disclosed.

公开(公告)号：US11568609B1

公开(公告)日：2023-01-31

申请号：US17315056

申请日：2021-05-07

Applicant: Meta Platforms Technologies, LLC

Inventor： Xinqiao Liu ,  Barbara De Salvo ,  Hans Reyserhove ,  Ziyun Li ,  Asif Imtiaz Khan ,  Syed Shakib Sarwar

IPC: G06T19/00 ,  G06F3/01 ,  G06T7/73 ,  G02B27/01 ,  G06V20/20 ,  G06K9/62 ,  G06V10/44 ,  G06V10/94 ,  G06V20/64 ,  G06V30/192 ,  H04N5/378 ,  H01L27/146 ,  H01L31/0352 ,  H01L31/0384 ,  H01L27/28 ,  H01L27/30

Abstract: In one example, an apparatus comprises: a first sensor layer, including an array of pixel cells configured to generate pixel data; and one or more semiconductor layers located beneath the first sensor layer with the one or more semiconductor layers being electrically connected to the first sensor layer via interconnects. The one or more semiconductor layers comprises on-chip compute circuits configured to receive the pixel data via the interconnects and process the pixel data, the on-chip compute circuits comprising: a machine learning (ML) model accelerator configured to implement a convolutional neural network (CNN) model to process the pixel data; a first memory to store coefficients of the CNN model and instruction codes; a second memory to store the pixel data of a frame; and a controller configured to execute the codes to control operations of the ML model accelerator, the first memory, and the second memory.

公开(公告)号：US20220408049A1

公开(公告)日：2022-12-22

申请号：US17728880

申请日：2022-04-25

Applicant: Meta Platforms Technologies, LLC

Inventor： Ziyun Li ,  Barbara De Salvo ,  Xinqiao Liu ,  Lyle David Bainbridge ,  Andrew Samuel Berkovich ,  Syed Shakib Sarwar ,  Song Chen ,  Tsung-Hsun Tsai

IPC: H04N5/369 ,  H01L27/146

Abstract: A stacked camera-image-sensor circuit may include (i) a first layer that includes a plurality of image sensing elements, (ii) a second layer that includes components that interface with the image sensing elements, and (iii) at least one additional layer that includes image-processing components. Various other methods, systems, and computer-readable media are also disclosed.

公开(公告)号：US12147596B1

公开(公告)日：2024-11-19

申请号：US17896679

申请日：2022-08-26

Applicant: Meta Platforms Technologies, LLC

Inventor： Qing Chao ,  Robert Dale Cavin ,  Guillaume Lestoquoy ,  Mohammadhossein Daraeihajitooei ,  Xinqiao Liu ,  Raffaele Capoccia ,  Sascha Hallstein ,  Ziyun Li

IPC: G06F3/01 ,  G06T7/73 ,  G06V10/25 ,  G06V10/60 ,  G06V40/18 ,  H04N5/33

Abstract: An eyebox region is illuminated with a fringe illumination pattern. An event sensor is configured to generate event-signals. Eye motion is determined from the event-signals. Eye-features are extracted from data generated by the event sensors and a predicted gaze vector is generated from the eye-features.

公开(公告)号：US11935575B1

公开(公告)日：2024-03-19

申请号：US17556436

申请日：2021-12-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Syed Shakib Sarwar ,  Ziyun Li ,  Xinqiao Liu ,  Barbara De Salvo

IPC: G11C11/16 ,  G06F1/16 ,  G11C11/404 ,  G11C11/412 ,  G11C11/54 ,  H03K19/017 ,  G11C11/4091 ,  G11C11/4096

CPC classification number: G11C11/4045 ,  G06F1/163 ,  G11C11/16 ,  G11C11/412 ,  G11C11/54 ,  H03K19/01742 ,  G11C11/4091 ,  G11C11/4096 ,  G02B27/017 ,  G02B2027/014 ,  G09G5/363 ,  G09G2360/18 ,  G06N3/08 ,  G06T1/60

Abstract: An example apparatus having a heterogenous memory system includes a first sensor layer, of a plurality of stacked sensor layers, including an array of pixels; and one or more semiconductor layers of the plurality of stacked sensor layers located beneath the first sensor layer, the one or more semiconductor layers configured to process pixel data output by the array of pixels, the one or more semiconductor layers including a first memory to store most significant bits (“MSBs”) of data involved in the processing of the pixel data; a second memory to store least significant bits (“LSBs”) of the data; and wherein the first memory has a lower bit error rate (“BER”) than the second memory.

公开(公告)号：US20220405553A1

公开(公告)日：2022-12-22

申请号：US17833402

申请日：2022-06-06

Applicant: Meta Platforms Technologies, LLC

Inventor： Ziyun Li ,  Xinqiao Liu ,  Yiming Gan

IPC: G06N3/04 ,  G06N3/063 ,  H04N5/351 ,  H04N5/378 ,  G06F9/50

Abstract: In one example, an apparatus comprises: a memory to store input data and weights, the input data comprising groups of data elements, each group being associated with a channel of channels, the weights comprising weight tensors, each weight tensor being associated with a channel of the channels; a data sparsity map generation circuit configured to generate, based on the input data, a channel sparsity map and a spatial sparsity map, the channel sparsity map indicating channels associated with first weights tensors to be selected, the spatial sparsity map indicating spatial locations of first data elements; a gating circuit configured to: fetch, based on the channel sparsity map and the sparsity map, the first weights tensors and the first data elements from the memory; and a processing circuit configured to perform neural network computations on the first data elements and the first weights tensors to generate a processing result.