公开(公告)号：US20210056749A1

公开(公告)日：2021-02-25

申请号：US16960163

申请日：2019-01-18

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Maxime Meilland ,  Patrick W. O'Keefe

IPC: G06T15/08 ,  G06T19/00 ,  G06T17/20 ,  G06F3/01 ,  G02B27/01

Abstract: In one implementation, a method includes: obtaining locality data characterizing objects and relative spatial information of a volumetric region around a user; synthesizing a mesh map of the volumetric region based on the locality data; selecting synthesized reality (SR) content based on the mesh map, wherein the SR content satisfies a dimensional variance threshold relative to one or more portions of the mesh map; compositing at least a portion of the SR content with the mesh map in order to generate composite SR content; and presenting the composite SR content to the user in order to occlude at least a portion of a visual presentation of the volumetric region. In some implementations, the SR content is adapted to fit the one or more portions of the mesh map. In some implementations, the SR content is updated as the user location changes or the user interacts with the SR content.

公开(公告)号：US20240233205A1

公开(公告)日：2024-07-11

申请号：US18612967

申请日：2024-03-21

Applicant: Apple Inc.

Inventor： Maxime Meilland ,  Duncan A. McRoberts ,  Julien Monat Rodier ,  Seyedkoosha Mirhosseini

IPC: G06T11/00 ,  G06T3/00 ,  G06T7/50

CPC classification number: G06T11/00 ,  G06T3/00 ,  G06T7/50

Abstract: In one implementation, a method of performing perspective correction of an image is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, an image of a physical environment. The method includes obtaining a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment. The method includes generating a clamped depth map of the image of the physical environment based on the plurality of depths, wherein each element of the clamped depth map has a depth value above or equal to a depth threshold. The method includes generating a display image by transforming, using the one or more processors, the image of the physical environment based on the clamped depth map and a difference between a perspective of the image sensor and a perspective of a user. The method includes displaying, on the display, the display image.

公开(公告)号：US11928779B2

公开(公告)日：2024-03-12

申请号：US17717393

申请日：2022-04-11

Applicant: Apple Inc.

Inventor： Maxime Meilland ,  Andrew Predoehl ,  Kyle L. Simek ,  Ming Chuang ,  Pedro A. Pinies Rodriguez

IPC: G06T17/20

CPC classification number: G06T17/20 ,  G06T3/4007 ,  G06T2200/04

Abstract: Various implementations disclosed herein generate a mesh representing the surfaces in a physical environment. The mesh is generated using multi-resolution voxels based on detected depth information, e.g., from a depth camera. The techniques may use multiple hash tables to store the multi-resolution voxel data. For example, the hash tables may store each voxel's 3D position and a truncated signed distance field (TSDF) value corresponding to each voxels' distance to a nearest surface. Each of the multiple hash tables may include data corresponding to a different level of resolution and those resolutions may depend upon distance/noise or other factors. For example, voxels close to a depth camera may have a finer resolution and smaller size compared to voxels that are further from the depth camera. Techniques disclosed herein may involve using a meshing algorithm that combines multi-resolution voxel information stored in multiple hash tables to generate a single mesh.

公开(公告)号：US20240005536A1

公开(公告)日：2024-01-04

申请号：US18212480

申请日：2023-06-21

Applicant: Apple Inc.

Inventor： Maxime Meilland ,  Emmanuel Piuze-Phaneuf

IPC: G06T7/50 ,  G06F3/04815 ,  H04N13/279 ,  G06V40/10

CPC classification number: G06T7/50 ,  G06F3/04815 ,  H04N13/279 ,  G06V40/107 ,  G06T2207/30196 ,  G06V2201/07

Abstract: In one implementation, a method of determining a display location is performed by a device including one or more processors and non-transitory memory. The method includes obtaining a camera set of two-dimensional coordinates of a user input object in a physical environment. The method includes obtaining depth information of the physical environment excluding the user input object. The method includes transforming the camera set of two-dimensional coordinates into a display set of two-dimensional coordinates based on the depth information of the physical environment excluding the user input object.

公开(公告)号：US20210366179A1

公开(公告)日：2021-11-25

申请号：US17397437

申请日：2021-08-09

Applicant: Apple Inc.

Inventor： Ian M. Richter ,  Maxime Meilland ,  Patrick W. O'Keefe

IPC: G06T15/08 ,  G02B27/01 ,  G06F3/01 ,  G06T17/20 ,  G06T19/00

Abstract: In one implementation, a method includes: obtaining locality data characterizing objects and relative spatial information of a volumetric region around a user; synthesizing a mesh map of the volumetric region based on the locality data; selecting synthesized reality (SR) content based on the mesh map, wherein the SR content satisfies a dimensional variance threshold relative to one or more portions of the mesh map; compositing at least a portion of the SR content with the mesh map in order to generate composite SR content; and presenting the composite SR content to the user in order to occlude at least a portion of a visual presentation of the volumetric region. In some implementations, the SR content is adapted to fit the one or more portions of the mesh map. In some implementations, the SR content is updated as the user location changes or the user interacts with the SR content.