公开(公告)号：US20240273841A1

公开(公告)日：2024-08-15

申请号：US18638995

申请日：2024-04-18

Applicant: Magic Leap, Inc.

Inventor： Ali Shahrokni ,  Daniel Olshansky ,  Xuan Zhao ,  Rafael Domingos Torres ,  Joel David Holder ,  Keng-Sheng Lin ,  Ashwin Swaminathan ,  Anush Mohan

IPC: G06T19/00 ,  G06F3/01 ,  G06T7/73 ,  H04L67/131

CPC classification number: G06T19/006 ,  G06F3/011 ,  G06T7/73 ,  G06T2200/04 ,  H04L67/131

Abstract: A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. The request may include one or more sets of feature descriptors extracted from an image of the physical world around the device. Those features may be posed relative to a coordinate frame used by the local device. The localization service may identify one or more stored maps with a matching set of features. Based on a transformation required to align the features from the device with the matching set of features, the localization service may compute and return to the device a transformation to relate its local coordinate frame to a coordinate frame of the stored map.

公开(公告)号：US11978159B2

公开(公告)日：2024-05-07

申请号：US17542306

申请日：2021-12-03

Applicant: Magic Leap, Inc.

Inventor： Anush Mohan ,  Rafael Domingos Torres ,  Daniel Olshansky ,  Samuel A. Miller ,  Jehangir Tajik ,  Joel David Holder ,  Jeremy Dwayne Miranda ,  Robert Blake Taylor ,  Ashwin Swaminathan ,  Lomesh Agarwal ,  Hiral Honar Barot ,  Helder Toshiro Suzuki ,  Ali Shahrokni ,  Eran Guendelman ,  Prateek Singhal ,  Xuan Zhao ,  Siddharth Choudhary ,  Nicholas Atkinson Kramer ,  Kenneth William Tossell ,  Christian Ivan Robert Moore

IPC: G06T17/05 ,  G06F3/01 ,  G06F16/29 ,  G06T7/73 ,  G06T19/00 ,  H04L67/01

CPC classification number: G06T17/05 ,  G06F3/012 ,  G06F16/29 ,  G06T7/74 ,  G06T19/006 ,  G06T2207/20216 ,  H04L67/01

Abstract: A cross reality system that provides an immersive user experience by storing persistent spatial information about the physical world that one or multiple user devices can access to determine position within the physical world and that applications can access to specify the position of virtual objects within the physical world. Persistent spatial information enables users to have a shared virtual, as well as physical, experience when interacting with the cross reality system. Further, persistent spatial information may be used in maps of the physical world, enabling one or multiple devices to access and localize into previously stored maps, reducing the need to map a physical space before using the cross reality system in it. Persistent spatial information may be stored as persistent coordinate frames, which may include a transformation relative to a reference orientation and information derived from images in a location corresponding to the persistent coordinate frame.

公开(公告)号：US11557099B2

公开(公告)日：2023-01-17

申请号：US17185706

申请日：2021-02-25

Applicant: Magic Leap, Inc.

Inventor： Ali Shahrokni ,  Keng-Sheng Lin ,  Xuan Zhao ,  Christian Ivan Robert Moore ,  Ashwin Swaminathan

IPC: G09G5/00 ,  G06T19/00 ,  G06T7/70 ,  H04L67/131

Abstract: A system form localizing an electronic device with dynamic buffering identifies, from the buffer, a first set of features that is extracted from a first image captured by the electronic device and receives, at the system, a second set of features that is extracted from a second image captured by the electronic device. The system further determines a first characteristic for the first set of features and a second characteristic for the second set of features and determines whether a triggering condition for dynamically changing a size of the buffer is satisfied based at least in part upon the first characteristic for the first set of features and the second characteristic for the second set of features.

公开(公告)号：US11257294B2

公开(公告)日：2022-02-22

申请号：US17071907

申请日：2020-10-15

Applicant: Magic Leap, Inc.

Inventor： Xuan Zhao ,  Ali Shahrokni ,  Daniel Olshansky ,  Christian Ivan Robert Moore ,  Rafael Domingos Torres ,  Joel David Holder

IPC: G06T19/00 ,  G06T7/73 ,  H04L67/131 ,  G06F3/01 ,  G06T5/50 ,  G06T5/00

Abstract: A cross reality system enables any of multiple types of devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. Devices of any type, with native hardware and software configured for augmented reality operations may be configured to work with the cross reality system by incorporating components that interface between the native AR framework of the device and the cloud-based localization service. These components may present position information about the device in a format recognized by the localization service. Additionally, these components may filter or otherwise process perception data provided by the native AR framework to increase the accuracy of localization.

公开(公告)号：US20210209859A1

公开(公告)日：2021-07-08

申请号：US17208844

申请日：2021-03-22

Applicant: Magic Leap, Inc.

Inventor： Jeremy Dwayne Miranda ,  Rafael Domingos Torres ,  Daniel Olshansky ,  Anush Mohan ,  Robert Blake Taylor ,  Samuel A. Miller ,  Jehangir Tajik ,  Ashwin Swaminathan ,  Lomesh Agarwal ,  Ali Shahrokni ,  Prateek Singhal ,  Joel David Holder ,  Xuan Zhao ,  Siddharth Choudhary ,  Helder Toshiro Suzuki ,  Hiral Honar Barot ,  Eran Guendelman ,  Michael Harold Liebenow ,  Christian Ivan Robert Moore

IPC: G06T19/00 ,  G02B27/00 ,  G02B27/01 ,  G06F3/01 ,  G06K9/00 ,  G06T19/20

Abstract: An augmented reality viewing system is described. A local coordinate frame of local content is transformed to a world coordinate frame. A further transformation is made to a head coordinate frame and a further transformation is made to a camera coordinate frame that includes all pupil positions of an eye. One or more users may interact in separate sessions with a viewing system. If a canonical map is available, the earlier map is downloaded onto a viewing device of a user. The viewing device then generates another map and localizes the subsequent map to the canonical map.

公开(公告)号：US20200051328A1

公开(公告)日：2020-02-13

申请号：US16593745

申请日：2019-10-04

Applicant: Magic Leap, Inc.

Inventor： Anush Mohan ,  Rafael Domingos Torres ,  Daniel Olshansky ,  Samuel A. Miller ,  Jehangir Tajik ,  Joel David Holder ,  Jeremy Dwayne Miranda ,  Robert Blake Taylor ,  Ashwin Swaminathan ,  Lomesh Agarwal ,  Hiral Honar Barot ,  Helder Toshiro Suzuki ,  Ali Shahrokni ,  Eran Guendelman ,  Prateek Singhal ,  Xuan Zhao ,  Siddharth Choudhary ,  Nick Kramer ,  Ken Tossell

IPC: G06T17/05 ,  G06F3/01 ,  G06T7/73 ,  G06T19/00 ,  G06F16/29

Abstract: A cross reality system that provides an immersive user experience by storing persistent spatial information about the physical world that one or multiple user devices can access to determine position within the physical world and that applications can access to specify the position of virtual objects within the physical world. Persistent spatial information enables users to have a shared virtual, as well as physical, experience when interacting with the cross reality system. Further, persistent spatial information may be used in maps of the physical world, enabling one or multiple devices to access and localize into previously stored maps, reducing the need to map a physical space before using the cross reality system in it. Persistent spatial information may be stored as persistent coordinate frames, which may include a transformation relative to a reference orientation and information derived from images in a location corresponding to the persistent coordinate frame.

公开(公告)号：US12293471B2

公开(公告)日：2025-05-06

申请号：US17571172

申请日：2022-01-07

Applicant: Magic Leap, Inc.

Inventor： Xuan Zhao ,  Ali Shahrokni ,  Daniel Olshansky ,  Christian Ivan Robert Moore ,  Rafael Domingos Torres ,  Joel David Holder

IPC: G06T19/00 ,  G06F3/01 ,  G06T5/50 ,  G06T5/70 ,  G06T7/73 ,  H04L67/131

Abstract: A cross reality system enables any of multiple types of devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. Devices of any type, with native hardware and software configured for augmented reality operations may be configured to work with the cross reality system by incorporating components that interface between the native AR framework of the device and the cloud-based localization service. These components may present position information about the device in a format recognized by the localization service. Additionally, these components may filter or otherwise process perception data provided by the native AR framework to increase the accuracy of localization.

公开(公告)号：US12283012B2

公开(公告)日：2025-04-22

申请号：US18510623

申请日：2023-11-15

Applicant: Magic Leap, Inc.

Inventor： Xuan Zhao ,  Christian Ivan Robert Moore ,  Sen Lin ,  Ali Shahrokni ,  Ashwin Swaminathan

IPC: G06T19/00 ,  G06F9/54 ,  G06F16/29 ,  G06F16/907

Abstract: A cross reality system enables any of multiple devices to efficiently access previously stored maps. Both stored maps and tracking maps used by portable devices may have any of multiple types of location metadata associated with them. The location metadata may be used to select a set of candidate maps for operations, such as localization or map merge, that involve finding a match between a location defined by location information from a portable device and any of a number of previously stored maps. The types of location metadata may prioritized for use in selecting the subset. To aid in selection of candidate maps, a universe of stored maps may be indexed based on geo-location information. A cross reality platform may update that index as it interacts with devices that supply geo-location information in connection with location information and may propagate that geo-location information to devices that do not supply it.

公开(公告)号：US12079382B2

公开(公告)日：2024-09-03

申请号：US18314289

申请日：2023-05-09

Applicant: MAGIC LEAP, INC.

Inventor： Divya Sharma ,  Ali Shahrokni ,  Anush Mohan ,  Prateek Singhal ,  Xuan Zhao ,  Sergiu Sima ,  Benjamin Langmann

IPC: G06T3/00 ,  G06F1/16 ,  G06F3/01 ,  G06T15/00 ,  G06T19/00 ,  G06V20/20

CPC classification number: G06F3/011 ,  G06F1/163 ,  G06T15/005 ,  G06T19/006 ,  G06V20/20

Abstract: An apparatus configured to be worn on a head of a user, includes: a screen configured to present graphics to the user; a camera system configured to view an environment in which the user is located; and a processing unit configured to determine a map based at least in part on output(s) from the camera system, wherein the map is configured for use by the processing unit to localize the user with respect to the environment; wherein the processing unit of the apparatus is also configured to obtain a metric indicating a likelihood of success to localize the user using the map, and wherein the processing unit is configured to obtain the metric by computing the metric or by receiving the metric.

公开(公告)号：US12002170B2

公开(公告)日：2024-06-04

申请号：US17894336

申请日：2022-08-24

Applicant: Magic Leap, Inc.

Inventor： Ali Shahrokni ,  Keng-Sheng Lin ,  Xuan Zhao ,  Christian Ivan Robert Moore ,  Ashwin Swaminathan

IPC: G06T19/00 ,  G06T7/70 ,  H04L67/131

CPC classification number: G06T19/006 ,  G06T7/70 ,  H04L67/131

Abstract: A system form localizing an electronic device with dynamic buffering identifies, from the buffer, a first set of features that is extracted from a first image captured by the electronic device and receives, at the system, a second set of features that is extracted from a second image captured by the electronic device. The system further determines a first characteristic for the first set of features and a second characteristic for the second set of features and determines whether a triggering condition for dynamically changing a size of the buffer is satisfied based at least in part upon the first characteristic for the first set of features and the second characteristic for the second set of features.