公开(公告)号：US20250067980A1

公开(公告)日：2025-02-27

申请号：US18946168

申请日：2024-11-13

Applicant: Apple Inc.

Inventor： Simon Fortin-Deschenes ,  Vincent Chapdelaine-Couture ,  Yan Cote ,  Anthony Ghannoum

IPC: G02B27/01 ,  G02B27/00 ,  G06T19/00

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

公开(公告)号：US20230324684A1

公开(公告)日：2023-10-12

申请号：US18209651

申请日：2023-06-14

Applicant: Apple Inc.

Inventor： Simon Fortin-Deschenes ,  Vincent Chapdelaine-Couture ,  Yan Cote ,  Anthony Ghannoum

IPC: G02B27/01 ,  G02B27/00 ,  G06T19/00

CPC classification number: G02B27/017 ,  G02B27/0093 ,  G06T19/006 ,  G02B2027/0134 ,  G02B2027/0138 ,  G02B2027/014 ,  G02B2027/0187

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

公开(公告)号：US12125123B2

公开(公告)日：2024-10-22

申请号：US18209643

申请日：2023-06-14

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Marc-Andre Chenier ,  Yan Cote ,  Yves Millette

IPC: G06T1/20 ,  G06T5/73 ,  G06T5/90 ,  G06T7/73 ,  G06T15/00 ,  G06T15/20 ,  G06T19/00

CPC classification number: G06T1/20 ,  G06T5/73 ,  G06T5/90 ,  G06T7/73 ,  G06T15/005 ,  G06T15/20 ,  G06T19/006

Abstract: In some implementations, a method includes: determining a complexity value for first image data associated with of a physical environment that corresponds to a first time period; determining an estimated composite setup time based on the complexity value for the first image data and virtual content for compositing with the first image data; in accordance with a determination that the estimated composite setup time exceeds the threshold time: forgoing rendering the virtual content from the perspective that corresponds to the camera pose of the device relative to the physical environment during the first time period; and compositing a previous render of the virtual content for a previous time period with the first image data to generate the graphical environment for the first time period.

公开(公告)号：US12094069B2

公开(公告)日：2024-09-17

申请号：US18233653

申请日：2023-08-14

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Sandy J. Carter ,  Vincent Chapdelaine-Couture ,  Marc-Andre Chenier ,  Yan Cote ,  Simon Fortin-Deschênes ,  Anthony Ghannoum ,  Tomlinson Holman ,  Marc-Olivier Lepage ,  Yves Millette

IPC: G06T19/00 ,  G06F3/01 ,  G06F3/16 ,  G06T5/00 ,  G06T11/00 ,  H04N13/117 ,  H04N13/156 ,  H04N13/194 ,  G02B27/01 ,  G06F7/32

CPC classification number: G06T19/006 ,  G06F3/011 ,  G06F3/165 ,  G06T5/00 ,  G06T11/00 ,  H04N13/117 ,  H04N13/156 ,  H04N13/194 ,  G02B2027/0138 ,  G02B27/0179 ,  G06F7/32

Abstract: The present disclosure relates to techniques for presenting a combined view of a virtual environment and a real environment in response to detecting a transition event associated with an object in the real environment. While presenting the combined view, if an input of a first type is detected, the combined view is adjusted by increasing the visibility of imagery of the virtual environment and decreasing the visibility of imagery of the real environment. If an input of a second type is detected, the combined view is adjusted by decreasing the visibility of the imagery of the virtual environment and increasing the visibility of the imagery of the real environment.

公开(公告)号：US11880911B2

公开(公告)日：2024-01-23

申请号：US17272025

申请日：2019-09-06

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Sandy J. Carter ,  Vincent Chapdelaine-Couture ,  Marc-Andre Chenier ,  Yan Cote ,  Simon Fortin-Deschênes ,  Anthony Ghannoum ,  Tomlinson Holman ,  Marc-Olivier Lepage ,  Yves Millette

IPC: G06T11/00 ,  G06F3/16 ,  H04N13/117 ,  H04N13/156 ,  H04N13/194 ,  G02B27/01 ,  G06F3/01

CPC classification number: G06T11/00 ,  G02B27/0179 ,  G06F3/011 ,  G06F3/165 ,  H04N13/117 ,  H04N13/156 ,  H04N13/194 ,  G02B2027/0138

Abstract: The present disclosure relates to techniques for transitioning between imagery and sounds of two different environments, such as a virtual environment and a real environment. A view of a first environment and audio associated with the first environment are provided. In response to detecting a transition event, a view of the first environment combined with a second environment is provided. The combined view includes imagery of the first environment at a first visibility value and imagery of the second environment at a second visibility value. In addition, in response to detecting a transition event, the first environment audio is mixed with audio associated with the second environment.

公开(公告)号：US11790569B2

公开(公告)日：2023-10-17

申请号：US17272251

申请日：2019-09-06

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Sandy J. Carter ,  Vincent Chapdelaine-Couture ,  Marc-Andre Chenier ,  Yan Cote ,  Simon Fortin-Deschênes ,  Anthony Ghannoum ,  Tomlinson Holman ,  Marc-Olivier Lepage ,  Yves Millette

IPC: G06T11/00 ,  G06F3/16 ,  H04N13/117 ,  H04N13/156 ,  H04N13/194 ,  G02B27/01 ,  G06F3/01

CPC classification number: G06T11/00 ,  G02B27/0179 ,  G06F3/011 ,  G06F3/165 ,  H04N13/117 ,  H04N13/156 ,  H04N13/194 ,  G02B2027/0138

Abstract: The present disclosure relates to techniques for inserting imagery from a real environment into a virtual environment. While presenting (e.g., displaying) the virtual environment at an electronic device, a proximity of the electronic device to a physical object located in a real environment is detected. In response to detecting that the proximity of the electronic device to the physical object is less than a first threshold distance, imagery of the physical object is isolated from other imagery of the real environment. The isolated imagery of the physical object is inserted into the virtual environment at a location corresponding to the location of the physical object in the real environment. The imagery of the physical object has a first visibility value associated with the proximity of the electronic device to the physical object.

公开(公告)号：US20230325965A1

公开(公告)日：2023-10-12

申请号：US18209643

申请日：2023-06-14

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Marc-Andre Chenier ,  Yan Cote ,  Yves Millette

IPC: G06T1/20 ,  G06T19/00 ,  G06T7/73 ,  G06T15/20 ,  G06T5/00 ,  G06T15/00

CPC classification number: G06T1/20 ,  G06T19/006 ,  G06T7/73 ,  G06T15/20 ,  G06T5/003 ,  G06T15/005 ,  G06T5/007

Abstract: In some implementations, a method includes: determining a complexity value for first image data associated with of a physical environment that corresponds to a first time period; determining an estimated composite setup time based on the complexity value for the first image data and virtual content for compositing with the first image data; in accordance with a determination that the estimated composite setup time exceeds the threshold time: forgoing rendering the virtual content from the perspective that corresponds to the camera pose of the device relative to the physical environment during the first time period; and compositing a previous render of the virtual content for a previous time period with the first image data to generate the graphical environment for the first time period.

公开(公告)号：US11736822B1

公开(公告)日：2023-08-22

申请号：US17176771

申请日：2021-02-16

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Marc-Andre Chenier ,  Yan Cote ,  Yves Millette

IPC: G06T1/60 ,  G06T1/20 ,  H04N23/80 ,  H04N5/351

CPC classification number: H04N5/351 ,  G06T1/20 ,  G06T1/60 ,  H04N23/80

Abstract: A method includes obtaining a first plurality of image frames that are provided by an image sensor at a first output rate. The method includes determining a GPU temporal processing value characterizing the GPU processing of the first plurality of image frames. The GPU temporal processing value is a function of an amount of information associated with the first plurality of image frames. The method includes determining an operation value associated with the image sensor based on a function of the GPU temporal processing value. The operation value modifies the operation of the image sensor. The method includes changing, using a controller, the operation of the image sensor as a function of the operation value so that the image sensor provides a second plurality of image frames at a second output rate.

公开(公告)号：US20250156985A1

公开(公告)日：2025-05-15

申请号：US18922265

申请日：2024-10-21

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Marc-Andre Chenier ,  Yan Cote ,  Yves Millette

IPC: G06T1/20 ,  G06T5/73 ,  G06T5/90 ,  G06T7/73 ,  G06T15/00 ,  G06T15/20 ,  G06T19/00

Abstract: In some implementations, a method includes: determining a complexity value for first image data associated with of a physical environment that corresponds to a first time period; determining an estimated composite setup time based on the complexity value for the first image data and virtual content for compositing with the first image data; in accordance with a determination that the estimated composite setup time exceeds the threshold time: forgoing rendering the virtual content from the perspective that corresponds to the camera pose of the device relative to the physical environment during the first time period; and compositing a previous render of the virtual content for a previous time period with the first image data to generate the graphical environment for the first time period.

公开(公告)号：US11704766B2

公开(公告)日：2023-07-18

申请号：US17979630

申请日：2022-11-02

Applicant: Apple Inc.

Inventor： Bertrand Nepveu ,  Marc-Andre Chenier ,  Yan Cote ,  Yves Millette

IPC: G06T1/20 ,  G06T19/00 ,  G06T7/73 ,  G06T15/20 ,  G06T5/00 ,  G06T15/00

CPC classification number: G06T1/20 ,  G06T5/003 ,  G06T5/007 ,  G06T7/73 ,  G06T15/005 ,  G06T15/20 ,  G06T19/006

Abstract: In some implementations, a method includes: determining a complexity value for first image data associated with of a physical environment that corresponds to a first time period; determining an estimated composite setup time based on the complexity value for the first image data and virtual content for compositing with the first image data; in accordance with a determination that the estimated composite setup time exceeds the threshold time: forgoing rendering the virtual content from the perspective that corresponds to the camera pose of the device relative to the physical environment during the first time period; and compositing a previous render of the virtual content for a previous time period with the first image data to generate the graphical environment for the first time period.