公开(公告)号：US20240114226A1

公开(公告)日：2024-04-04

申请号：US17564889

申请日：2021-12-29

Applicant: Meta Platforms Technologies, LLC

Inventor： Andrew Samuel Berkovich ,  Warren Andrew Hunt ,  Daniel Morgan ,  Shlomo Alkalay ,  Jack Thomas Lavier

IPC: H04N5/225 ,  G06F3/01 ,  G06V10/25 ,  G09G3/32 ,  H01L23/00 ,  H01L27/146 ,  H01L27/15 ,  H04N5/232

CPC classification number: H04N5/2257 ,  G06F3/013 ,  G06V10/25 ,  G09G3/32 ,  H01L24/08 ,  H01L24/16 ,  H01L27/14618 ,  H01L27/14634 ,  H01L27/14636 ,  H01L27/156 ,  H04N5/2252 ,  H04N5/232 ,  G09G2360/18 ,  H01L2224/08145 ,  H01L2224/16145

Abstract: In one example, an apparatus for integrating sensing and display system includes a first semiconductor layer that includes an image sensor; a second semiconductor layer that includes a display; a third semiconductor layer that includes compute circuits configured to support an image sensing operation by the image sensor and a display operation by the display; and a semiconductor package that encloses the first, second, and third semiconductor layers, the semiconductor package further including a first opening to expose the image sensor and a second opening to expose the display. The first, second, and third semiconductor layers form a first stack structure along a first axis. The third semiconductor layer is sandwiched between the first semiconductor layer and the second semiconductor layer in the first stack structure.

公开(公告)号：US11910114B2

公开(公告)日：2024-02-20

申请号：US17364763

申请日：2021-06-30

Applicant: Meta Platforms Technologies, LLC

Inventor： Andrew Samuel Berkovich

IPC: H04N25/75 ,  H04N25/79 ,  H04N25/709

CPC classification number: H04N25/75 ,  H04N25/709 ,  H04N25/79

Abstract: In some examples, an apparatus comprises: an array of pixel cells, each pixel cell of the array of pixel cells configured to output a voltage; first analog-to-digital converters (ADCs), each first ADC being associated with a pixel cell or a block of pixel cells; second ADCs, each second ADC being associated with a column of pixel cells of the array of pixel cells; and a controller configured to: in a first mode, enable at least a subset of the first ADCs to perform a quantization operation of the voltages output by at least a subset of the pixel cells in parallel to generate a first image frame; and in a second mode, enable at least a subset of the second ADCs to perform quantization operations sequentially of the voltages output by pixel cells within at least a subset of the columns of pixel cells to generate a second image frame.

公开(公告)号：US20230119703A1

公开(公告)日：2023-04-20

申请号：US18069896

申请日：2022-12-21

Applicant: Meta Platforms Technologies, LLC

Inventor： Steven John Lovegrove ,  Richard Andrew Newcombe ,  Andrew Samuel Berkovich ,  Lingni Ma ,  Chao Li

IPC: G06T17/10 ,  G06T19/00 ,  G06T7/00 ,  G06T17/20

Abstract: In one embodiment, a computing system instructs, at a first time, a camera having a plurality of pixel sensors to use the plurality of pixel sensors to capture a first image of an environment comprising an object. The computing system predicts, using at least the first image, a projection of the object appearing in a virtual image plane associated with a predicted camera pose at a second time. The computing system determines, based on the predicted projection of the object, a first region of pixels and a second region of pixels. The computing system generates pixel-activation instructions for the first region of pixels and the second region of pixels. The computing system instructs the camera to capture a second image of the environment at the second time according to the pixel-activation instructions. The pixel-activation instructions are configured to cause a first subset of the plurality of pixel sensors to sample the first region of pixels and a second subset of the plurality of pixel sensors to sample the second region of pixels. The first subset of the plurality of pixel sensors used for sampling the first region of pixels is more dense than the second subset of the plurality of pixel sensors used for sampling the second region of pixels. The computing system tracks, based on the second image, the object at the second time.

公开(公告)号：US20230032925A1

公开(公告)日：2023-02-02

申请号：US17729951

申请日：2022-04-26

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Ziyun Li ,  Barbara De Salvo ,  Xinqiao Liu ,  Lyle David Bainbridge ,  Andrew Samuel Berkovich ,  Syed Shakib Sarwar ,  Song Chen ,  Tsung-Hsun Tsai

IPC: H04N5/232 ,  G06V10/147 ,  G06V10/94 ,  G06V10/10 ,  G06T7/20 ,  G06V10/70

Abstract: The disclosed system may include a first layer that includes multiple digital pixel sensors configured to detect light. The system may also include a second layer that includes various image processing components configured to process the light detected by the digital pixel sensors. Still further, the system may include a third layer that includes machine learning (ML) hardware processing components. The image processing components of the second layer may be communicatively connected to the ML hardware processing components of the third layer via multiple micro through-silicon vias (uTSVs). Various other methods of manufacturing, apparatuses, and computer-readable media are also disclosed.

公开(公告)号：US20240118423A1

公开(公告)日：2024-04-11

申请号：US18484762

申请日：2023-10-11

Applicant: Meta Platforms Technologies, LLC

Inventor： Onur Can Akkaya ,  Song Chen ,  Xinqiao Liu ,  Andrew Samuel Berkovich ,  Tsung-Hsun Tsai ,  Yi Zheng ,  Raffaele Capoccia

IPC: G01S17/894 ,  G01S7/487 ,  G01S17/42

CPC classification number: G01S17/894 ,  G01S7/4873 ,  G01S17/42

Abstract: A projector sequentially generates light pulses to form a sparse grid array. The light pulses reflect off objects in an environment and are reflected towards a sensor array. The sensor array sequentially senses the reflected light pulses across pixels of the sensor array. A depth sensing system calculates depth information of objects in the environment based on the sequential nature of the pulse generation and the pulse sensing. The depth sensing system calculates depth information based on both the positional and temporal information of generated light pulses, and the positional and temporal information of sensed light pulses. The depth sensing system may generate a representation of the environment based on the depth information.

公开(公告)号：US20240114263A1

公开(公告)日：2024-04-04

申请号：US18479765

申请日：2023-10-02

Applicant: Meta Platforms Technologies, LLC

Inventor： Andrew Samuel Berkovich ,  Xinqiao Liu ,  Onur Can Akkaya ,  Tsung-Hsun Tsai ,  Song Chen ,  Lyle David Bainbridge

IPC: H04N25/772 ,  H04N25/531 ,  H04N25/532

CPC classification number: H04N25/772 ,  H04N25/531 ,  H04N25/532

Abstract: An image-sensing system includes multiple sensing modules. Each of the multiple sensing modules includes multiple optical sensors arranged in an array. Each of the multiple sensors is configured to be switched on and off to generate analog sensing data. The image-sensing system also includes an analog-to-digital converter (ADC) shared by the multiple optical sensors configured to convert analog sensing data generated by the multiple optical sensors into digital data. The image-sensing system also includes a processor configured to control the multiple sensing modules.

公开(公告)号：US11948089B2

公开(公告)日：2024-04-02

申请号：US17091331

申请日：2020-11-06

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Andrew Samuel Berkovich ,  Reid Pinkham

IPC: H04N5/232 ,  G06N3/04 ,  G06N3/08 ,  G06N3/084 ,  G06V10/141 ,  G06V10/147 ,  G06V10/94 ,  G06V20/20 ,  H04N23/61

CPC classification number: G06N3/084 ,  G06N3/04 ,  G06N3/08 ,  G06V10/141 ,  G06V10/147 ,  G06V10/95 ,  G06V10/955 ,  G06V20/20 ,  H04N23/61

Abstract: In one example, an apparatus comprises: an image sensor comprising a plurality of pixel cells; a frame buffer; and a sensor compute circuit configured to: receive, from the frame buffer, a first image frame comprising first active pixels and first inactive pixels, the first active pixels being generated by a first subset of the pixel cells selected based on first programming data; perform an image-processing operation on a first subset of pixels of the first image frame, whereby a second subset of pixels of the first image frame are excluded from the image-processing operation, to generate a processing output; based on the processing output, generate second programming data; and transmit the second programming data to the image sensor to select a second subset of the pixel cells to generate second active pixels for a second image frame.

公开(公告)号：US20240007771A1

公开(公告)日：2024-01-04

申请号：US18208236

申请日：2023-06-09

Applicant: Meta Platforms Technologies, LLC

Inventor： Andrew Samuel Berkovich ,  Jack Thomas Lavier ,  Reid Frederick Pinkham ,  Shlomo Alkalay

IPC: H04N25/78 ,  H04N25/40

CPC classification number: H04N25/78 ,  H04N25/40

Abstract: Systems and readout methods for foveated sensing may include a pixel array, readout circuitry, and processing logic. The pixel array may have a number of pixels. The readout circuitry may be configured to read image data from the pixel array for each of the plurality of pixels. The processing logic may be configured to identify a number of regions of interest (ROIs) within the pixel array. The processing logic may be configured to associate the image data for the pixels with one or more ROIs. The processing logic may be configured to arrange the image data into data frames. The data frames may include the image data ordered by ROI. Image data for inactive pixels may be removed from the image data and data frames prior to transmission. The processing logic may be configured to transmit the data frames in an order that is based on the ROIs.

公开(公告)号：US11863886B2

公开(公告)日：2024-01-02

申请号：US17591300

申请日：2022-02-02

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Andrew Samuel Berkovich ,  Xinqiao Liu ,  Song Chen ,  Nicholas Daniel Trail ,  Michael Patrick Schaub

IPC: H04N25/58 ,  H03M1/18 ,  H04N25/59

CPC classification number: H04N25/58 ,  H03M1/18 ,  H04N25/59

Abstract: In one example, an apparatus comprises: multiple distinct sets of photodiodes, wherein each set of photodiodes includes one or more photodiodes, one or more charge sensing units, and a controller. The controller is configured to: transfer charge generated by the one or more photodiodes in response to a different component of incident light to the one or more charge sensing units in order to convert the charge to voltages; perform one or more quantization processes of a plurality of quantization processes corresponding to a plurality of intensity ranges, wherein the one or more quantization processes quantizes the voltages from the one or more charge sensing units to digital values representing components of a pixel of different wavelength ranges; and generate a pixel value based on the at least some of the digital values.

公开(公告)号：US20220408049A1

公开(公告)日：2022-12-22

申请号：US17728880

申请日：2022-04-25

Applicant: Meta Platforms Technologies, LLC

Inventor： Ziyun Li ,  Barbara De Salvo ,  Xinqiao Liu ,  Lyle David Bainbridge ,  Andrew Samuel Berkovich ,  Syed Shakib Sarwar ,  Song Chen ,  Tsung-Hsun Tsai

IPC: H04N5/369 ,  H01L27/146

Abstract: A stacked camera-image-sensor circuit may include (i) a first layer that includes a plurality of image sensing elements, (ii) a second layer that includes components that interface with the image sensing elements, and (iii) at least one additional layer that includes image-processing components. Various other methods, systems, and computer-readable media are also disclosed.