公开(公告)号：US12099327B2

公开(公告)日：2024-09-24

申请号：US17360693

申请日：2021-06-28

Applicant: Meta Platforms Technologies, LLC

Inventor： Albert Parra Pozo ,  Joseph Virskus ,  Ganesh Venkatesh ,  Kai Li ,  Shen-Chi Chen ,  Amit Kumar ,  Rakesh Ranjan ,  Brian Keith Cabral ,  Samuel Alan Johnson ,  Wei Ye ,  Michael Alexander Snower ,  Yash Patel

IPC: G03H1/00 ,  G06N20/00 ,  G06T7/194 ,  G06T7/521 ,  G06T7/55 ,  G06T17/20 ,  G06T19/00 ,  G06V40/16 ,  H04M3/56 ,  H04N19/597 ,  G03H1/02

CPC classification number: G03H1/0005 ,  G06N20/00 ,  G06T7/194 ,  G06T7/521 ,  G06T7/55 ,  G06T17/20 ,  G06T19/006 ,  G06V40/176 ,  H04M3/567 ,  H04M3/568 ,  H04N19/597 ,  G03H2001/0088 ,  G03H2001/0204 ,  H04M2203/359

Abstract: A holographic calling system can capture and encode holographic data at a sender-side of a holographic calling pipeline and decode and present the holographic data as a 3D representation of a sender at a receiver-side of the holographic calling pipeline. The holographic calling pipeline can include stages to capture audio, color images, and depth images; densify the depth images to have a depth value for each pixel while generating parts masks and a body model; use the masks to segment the images into parts needed for hologram generation; convert depth images into a 3D mesh; paint the 3D mesh with color data; perform torso disocclusion; perform face reconstruction; and perform audio synchronization. In various implementations, different of these stages can be performed sender-side or receiver side. The holographic calling pipeline also includes sender-side compression, transmission over a communication channel, and receiver-side decompression and hologram output.

公开(公告)号：US20250045866A1

公开(公告)日：2025-02-06

申请号：US18365540

申请日：2023-08-04

Applicant: Meta Platforms Technologies, LLC

Inventor： Yuchen Fan ,  Bo Zhu ,  Rakesh Ranjan ,  Touraj Tajbakhsh

IPC: G06T3/40

Abstract: In one embodiment, a method by a computing system associated with an image sensor including a quad-Bayer color filter array includes accessing image-sensor data generated by the image sensor, where the quad-Bayer color filter array comprises sixteen sets of filters, each corresponding to a pixel location within a quad-Bayer pattern, splitting the image-sensor data into sixteen packed channels, where each of the sixteen packed channels corresponds to one of the sixteen sets of filters, producing three channels of interpolated pixels by processing the sixteen packed channels using a machine-learning model, where the three channels comprise red, green, and blue channels, and constructing an output image using the three channels of the interpolated pixels.

公开(公告)号：US11461962B1

公开(公告)日：2022-10-04

申请号：US17360758

申请日：2021-06-28

Applicant: Meta Platforms Technologies, LLC

Inventor： Albert Parra Pozo ,  Joseph Virskus ,  Ganesh Venkatesh ,  Kai Li ,  Shen-Chi Chen ,  Amit Kumar ,  Rakesh Ranjan ,  Brian Keith Cabral ,  Samuel Alan Johnson ,  Wei Ye ,  Michael Alexander Snower ,  Yash Patel

IPC: G06T17/00 ,  G06T9/00 ,  G06T11/00

Abstract: A holographic calling system can capture and encode holographic data at a sender-side of a holographic calling pipeline and decode and present the holographic data as a 3D representation of a sender at a receiver-side of the holographic calling pipeline. The holographic calling pipeline can include stages to capture audio, color images, and depth images; densify the depth images to have a depth value for each pixel while generating parts masks and a body model; use the masks to segment the images into parts needed for hologram generation; convert depth images into a 3D mesh; paint the 3D mesh with color data; perform torso disocclusion; perform face reconstruction; and perform audio synchronization. In various implementations, different of these stages can be performed sender-side or receiver side. The holographic calling pipeline also includes sender-side compression, transmission over a communication channel, and receiver-side decompression and hologram output.

公开(公告)号：US20250138880A1

公开(公告)日：2025-05-01

申请号：US18925225

申请日：2024-10-24

Applicant: Meta Platforms Technologies, LLC

Inventor： Sreyas Mohan ,  Rakesh Ranjan ,  Yilei Li

IPC: G06F9/48 ,  G06F11/30

Abstract: A method for signal processing includes receiving a definition associated with a task, and based on the definition, selecting a first sensor modality having a first operational cost from multiple sensor modalities each associated with respective operational costs. The method further includes capturing a first sensor signal using the first sensor modality, and making a first determination that the first sensor signal is not sufficient to perform the task. The method further includes selecting, from the sensor modalities, a second sensor modality having a second operational cost, the second operational cost being higher than the first operational cost and capturing a second sensor signal using the second sensor modality. The method further includes making a second determination that the second sensor signal may be used to perform the task and using the second sensor signal to perform the task.

公开(公告)号：US20250046301A1

公开(公告)日：2025-02-06

申请号：US18787739

申请日：2024-07-29

Applicant: Meta Platforms Technologies, LLC

Inventor： Balasubramanian Sivakumar ,  Min Hyuk Choi ,  Gang Chen ,  Gyungmin Kim ,  Ahmet Tura ,  Hai Jung In ,  Weiwei Wang ,  Suhui Lee ,  Woong Hwang ,  Wanyue Song ,  Wai Sze Tiffany Lam ,  Wanli Chi ,  Nikolaos Sarafianos ,  Rakesh Ranjan ,  Alexander Sorkine Hornung ,  Seonghyeon Nam ,  Hyunyoung Jung ,  Ruiming Xie ,  Ju Lin ,  Niko Moritz ,  Frank Torsten Bernd Seide

IPC: G10L15/16 ,  G02B6/26 ,  G02B6/27 ,  G02B6/34 ,  G06F40/40 ,  G10L15/02 ,  G10L15/18 ,  H10K59/12

Abstract: A display device includes a diffusion layer providing a single well, an optical element configured to receive light and output the received image light including a concentric series of lens segments made up of active facets, and a waveguide having an in-coupling grating and a waveguide body optically coupled to the image light, where a polarizing component is disposed over an output surface of the optical element. A method to improve a yield of silicon backplane displays includes connecting a diode having a resolution to a backplane and repairing a defect on a pixel display area by dividing the pixel display area into at least one subcircuit, while improving geometry stylization transfer for 3D models of real-world environments. A method for suppressing crosstalk for user conversations includes receiving multiple speech signals captured by multiple microphones and generating directional data for the multiple speech signals based on spatial filtering.