公开(公告)号：US11496723B1

公开(公告)日：2022-11-08

申请号：US16581074

申请日：2019-09-24

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Yan Cote

IPC: H04N13/275 ,  G06T19/00 ,  H04N13/349 ,  H04N13/282 ,  H04N13/383

Abstract: Generating a representation of a scene includes detecting an indication to capture sensor data to generate a virtual representation of a scene in a physical environment at a first time, in response to the indication obtaining first sensor data from a first capture device at the first time, obtaining second sensor data from a second capture device at the first time, and combining the first sensor data and the second sensor data to generate the virtual representation of the scene.

公开(公告)号：US12169276B2

公开(公告)日：2024-12-17

申请号：US18209651

申请日：2023-06-14

Applicant: Apple Inc.

Inventor： Simon Fortin-Deschenes ,  Vincent Chapdelaine-Couture ,  Yan Cote ,  Anthony Ghannoum

IPC: G02B27/01 ,  G02B27/00 ,  G06T19/00

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

公开(公告)号：US11778154B2

公开(公告)日：2023-10-03

申请号：US16637087

申请日：2018-08-08

Applicant: Apple Inc.

Inventor： Vincent Chapdelaine-Couture ,  Anthony Ghannoum ,  Yan Cote ,  Irving Lustigman ,  Marc-Andre Chenier ,  Simon Fortin-Deschenes ,  Bertrand Nepveu

IPC: G06T19/00 ,  H04N9/75 ,  H04N13/344 ,  H04N5/265

CPC classification number: H04N9/75 ,  G06T19/006 ,  H04N5/265 ,  H04N13/344

Abstract: A Head-Mounted Display with camera sensors to perform chroma keying in a mixed reality context is presented. Low latency is achieved by embedding the processing in the HMD itself, specifically, format camera images, detect the selected color range and make a composite with the virtual content.

公开(公告)号：US20230053205A1

公开(公告)日：2023-02-16

申请号：US17979630

申请日：2022-11-02

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Marc-Andre Chenier ,  Yan Cote ,  Yves Millette

IPC: G06T1/20 ,  G06T19/00 ,  G06T15/00 ,  G06T7/73 ,  G06T5/00 ,  G06T15/20

Abstract: In some implementations, a method includes: determining a complexity value for first image data associated with of a physical environment that corresponds to a first time period; determining an estimated composite setup time based on the complexity value for the first image data and virtual content for compositing with the first image data; in accordance with a determination that the estimated composite setup time exceeds the threshold time: forgoing rendering the virtual content from the perspective that corresponds to the camera pose of the device relative to the physical environment during the first time period; and compositing a previous render of the virtual content for a previous time period with the first image data to generate the graphical environment for the first time period.

公开(公告)号：US11521291B1

公开(公告)日：2022-12-06

申请号：US17215151

申请日：2021-03-29

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Marc-Andre Chenier ,  Yan Cote ,  Yves Millette

IPC: G06T1/20 ,  G06T19/00 ,  G06T7/73 ,  G06T15/20 ,  G06T5/00 ,  G06T15/00

Abstract: In some implementations, a method of reducing latency associated with an image read-out operation is performed at a device including one or more processors, non-transitory memory, an image processing architecture, and an image capture device. The method includes: obtaining first image data corresponding to a physical environment; reading a first slice of the first image data into an input buffer; performing processing operations on the first slice of the first image data to obtain a first portion of second image data; reading a second slice of the first image data into the input buffer; performing the image processing operations on the second slice of the first image data to obtain a second portion of the second image data; and generating an image frame of the physical environment based at least in part on the first and second portions of the second image data.