公开(公告)号：US20240273841A1

公开(公告)日：2024-08-15

申请号：US18638995

申请日：2024-04-18

申请人： Magic Leap, Inc.

发明人： Ali Shahrokni ,  Daniel Olshansky ,  Xuan Zhao ,  Rafael Domingos Torres ,  Joel David Holder ,  Keng-Sheng Lin ,  Ashwin Swaminathan ,  Anush Mohan

IPC分类号： G06T19/00 ,  G06F3/01 ,  G06T7/73 ,  H04L67/131

CPC分类号： G06T19/006 ,  G06F3/011 ,  G06T7/73 ,  G06T2200/04 ,  H04L67/131

摘要： A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. The request may include one or more sets of feature descriptors extracted from an image of the physical world around the device. Those features may be posed relative to a coordinate frame used by the local device. The localization service may identify one or more stored maps with a matching set of features. Based on a transformation required to align the features from the device with the matching set of features, the localization service may compute and return to the device a transformation to relate its local coordinate frame to a coordinate frame of the stored map.

公开(公告)号：US20240221306A1

公开(公告)日：2024-07-04

申请号：US18609101

申请日：2024-03-19

申请人： Magic Leap, Inc.

发明人： Elad Joseph ,  Gal Braun ,  Ali Shahrokni

IPC分类号： G06T17/00 ,  G06T15/00 ,  G06V10/40 ,  G06V10/46 ,  G06V20/20 ,  G06V40/18 ,  H04N23/57 ,  H04N23/90

CPC分类号： G06T17/00 ,  G06T15/005 ,  G06V10/40 ,  G06V10/462 ,  G06V20/20 ,  G06V40/18 ,  H04N23/57 ,  H04N23/90

摘要： A distributed, cross reality system efficiently and accurately compares location information that includes image frames. Each of the frames may be represented as a numeric descriptor that enables identification of frames with similar content. The resolution of the descriptors may vary for different computing devices in the distributed system based on degree of ambiguity in image comparisons and/or computing resources for the device. A descriptor computed for a cloud-based component operating on maps of large areas that can result in ambiguous identification of multiple image frames may use high resolution descriptors. High resolution descriptors reduce computationally intensive disambiguation processing. A portable device, which is more likely to operate on smaller maps and less likely to have the computational resources to compute a high resolution descriptor, may use a lower resolution descriptor.

公开(公告)号：US11978159B2

公开(公告)日：2024-05-07

申请号：US17542306

申请日：2021-12-03

申请人： Magic Leap, Inc.

发明人： Anush Mohan ,  Rafael Domingos Torres ,  Daniel Olshansky ,  Samuel A. Miller ,  Jehangir Tajik ,  Joel David Holder ,  Jeremy Dwayne Miranda ,  Robert Blake Taylor ,  Ashwin Swaminathan ,  Lomesh Agarwal ,  Hiral Honar Barot ,  Helder Toshiro Suzuki ,  Ali Shahrokni ,  Eran Guendelman ,  Prateek Singhal ,  Xuan Zhao ,  Siddharth Choudhary ,  Nicholas Atkinson Kramer ,  Kenneth William Tossell ,  Christian Ivan Robert Moore

IPC分类号： G06T17/05 ,  G06F3/01 ,  G06F16/29 ,  G06T7/73 ,  G06T19/00 ,  H04L67/01

CPC分类号： G06T17/05 ,  G06F3/012 ,  G06F16/29 ,  G06T7/74 ,  G06T19/006 ,  G06T2207/20216 ,  H04L67/01

摘要： A cross reality system that provides an immersive user experience by storing persistent spatial information about the physical world that one or multiple user devices can access to determine position within the physical world and that applications can access to specify the position of virtual objects within the physical world. Persistent spatial information enables users to have a shared virtual, as well as physical, experience when interacting with the cross reality system. Further, persistent spatial information may be used in maps of the physical world, enabling one or multiple devices to access and localize into previously stored maps, reducing the need to map a physical space before using the cross reality system in it. Persistent spatial information may be stored as persistent coordinate frames, which may include a transformation relative to a reference orientation and information derived from images in a location corresponding to the persistent coordinate frame.

公开(公告)号：US11557099B2

公开(公告)日：2023-01-17

申请号：US17185706

申请日：2021-02-25

申请人： Magic Leap, Inc.

发明人： Ali Shahrokni ,  Keng-Sheng Lin ,  Xuan Zhao ,  Christian Ivan Robert Moore ,  Ashwin Swaminathan

IPC分类号： G09G5/00 ,  G06T19/00 ,  G06T7/70 ,  H04L67/131

摘要： A system form localizing an electronic device with dynamic buffering identifies, from the buffer, a first set of features that is extracted from a first image captured by the electronic device and receives, at the system, a second set of features that is extracted from a second image captured by the electronic device. The system further determines a first characteristic for the first set of features and a second characteristic for the second set of features and determines whether a triggering condition for dynamically changing a size of the buffer is satisfied based at least in part upon the first characteristic for the first set of features and the second characteristic for the second set of features.

公开(公告)号：US11257294B2

公开(公告)日：2022-02-22

申请号：US17071907

申请日：2020-10-15

申请人： Magic Leap, Inc.

发明人： Xuan Zhao ,  Ali Shahrokni ,  Daniel Olshansky ,  Christian Ivan Robert Moore ,  Rafael Domingos Torres ,  Joel David Holder

IPC分类号： G06T19/00 ,  G06T7/73 ,  H04L67/131 ,  G06F3/01 ,  G06T5/50 ,  G06T5/00

摘要： A cross reality system enables any of multiple types of devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. Devices of any type, with native hardware and software configured for augmented reality operations may be configured to work with the cross reality system by incorporating components that interface between the native AR framework of the device and the cloud-based localization service. These components may present position information about the device in a format recognized by the localization service. Additionally, these components may filter or otherwise process perception data provided by the native AR framework to increase the accuracy of localization.

公开(公告)号：US20210209859A1

公开(公告)日：2021-07-08

申请号：US17208844

申请日：2021-03-22

申请人： Magic Leap, Inc.

发明人： Jeremy Dwayne Miranda ,  Rafael Domingos Torres ,  Daniel Olshansky ,  Anush Mohan ,  Robert Blake Taylor ,  Samuel A. Miller ,  Jehangir Tajik ,  Ashwin Swaminathan ,  Lomesh Agarwal ,  Ali Shahrokni ,  Prateek Singhal ,  Joel David Holder ,  Xuan Zhao ,  Siddharth Choudhary ,  Helder Toshiro Suzuki ,  Hiral Honar Barot ,  Eran Guendelman ,  Michael Harold Liebenow ,  Christian Ivan Robert Moore

IPC分类号： G06T19/00 ,  G02B27/00 ,  G02B27/01 ,  G06F3/01 ,  G06K9/00 ,  G06T19/20

摘要： An augmented reality viewing system is described. A local coordinate frame of local content is transformed to a world coordinate frame. A further transformation is made to a head coordinate frame and a further transformation is made to a camera coordinate frame that includes all pupil positions of an eye. One or more users may interact in separate sessions with a viewing system. If a canonical map is available, the earlier map is downloaded onto a viewing device of a user. The viewing device then generates another map and localizes the subsequent map to the canonical map.

公开(公告)号：US20200051328A1

公开(公告)日：2020-02-13

申请号：US16593745

申请日：2019-10-04

申请人： Magic Leap, Inc.

发明人： Anush Mohan ,  Rafael Domingos Torres ,  Daniel Olshansky ,  Samuel A. Miller ,  Jehangir Tajik ,  Joel David Holder ,  Jeremy Dwayne Miranda ,  Robert Blake Taylor ,  Ashwin Swaminathan ,  Lomesh Agarwal ,  Hiral Honar Barot ,  Helder Toshiro Suzuki ,  Ali Shahrokni ,  Eran Guendelman ,  Prateek Singhal ,  Xuan Zhao ,  Siddharth Choudhary ,  Nick Kramer ,  Ken Tossell

IPC分类号： G06T17/05 ,  G06F3/01 ,  G06T7/73 ,  G06T19/00 ,  G06F16/29

摘要： A cross reality system that provides an immersive user experience by storing persistent spatial information about the physical world that one or multiple user devices can access to determine position within the physical world and that applications can access to specify the position of virtual objects within the physical world. Persistent spatial information enables users to have a shared virtual, as well as physical, experience when interacting with the cross reality system. Further, persistent spatial information may be used in maps of the physical world, enabling one or multiple devices to access and localize into previously stored maps, reducing the need to map a physical space before using the cross reality system in it. Persistent spatial information may be stored as persistent coordinate frames, which may include a transformation relative to a reference orientation and information derived from images in a location corresponding to the persistent coordinate frame.

公开(公告)号：US12100108B2

公开(公告)日：2024-09-24

申请号：US17084174

申请日：2020-10-29

申请人： Magic Leap, Inc.

发明人： Ali Shahrokni ,  Rafael Domingos Torres ,  Joao Lucas Guberman Raza

IPC分类号： G06T19/00 ,  G06T3/40

CPC分类号： G06T19/006 ,  G06T3/40

摘要： A cross reality system that provides an immersive user experience shared by multiple user devices by providing quality information about a shared map. The quality information may be specific to individual user devices rendering virtual content specified with respect to the shared map. The quality information may be provided for persistent coordinate frames (PCFs) in the map. The quality information about a PCF may indicate positional uncertainty of virtual content, specified with respect to the PCF, when rendered on the user device. The quality information may be computed as upper bounding errors by determining error statistics for one or more steps in a process of specifying position with respect to the PCF or transforming that positional expression to a coordinate frame local to the device for rendering the virtual content. Applications running on individual user devices may adjust the rendering of virtual content based on the quality information about the shared map.

公开(公告)号：US12079382B2

公开(公告)日：2024-09-03

申请号：US18314289

申请日：2023-05-09

申请人： MAGIC LEAP, INC.

发明人： Divya Sharma ,  Ali Shahrokni ,  Anush Mohan ,  Prateek Singhal ,  Xuan Zhao ,  Sergiu Sima ,  Benjamin Langmann

IPC分类号： G06T3/00 ,  G06F1/16 ,  G06F3/01 ,  G06T15/00 ,  G06T19/00 ,  G06V20/20

CPC分类号： G06F3/011 ,  G06F1/163 ,  G06T15/005 ,  G06T19/006 ,  G06V20/20

摘要： An apparatus configured to be worn on a head of a user, includes: a screen configured to present graphics to the user; a camera system configured to view an environment in which the user is located; and a processing unit configured to determine a map based at least in part on output(s) from the camera system, wherein the map is configured for use by the processing unit to localize the user with respect to the environment; wherein the processing unit of the apparatus is also configured to obtain a metric indicating a likelihood of success to localize the user using the map, and wherein the processing unit is configured to obtain the metric by computing the metric or by receiving the metric.

公开(公告)号：US12008686B2

公开(公告)日：2024-06-11

申请号：US18057615

申请日：2022-11-21

申请人： Magic Leap, Inc.

发明人： Griffith Buckley Hazen ,  Amy DeDonato ,  Ali Shahrokni ,  Ben Weisbih ,  Vinayram Balakumar

IPC分类号： G06F3/01 ,  G06T3/60 ,  G06T11/00 ,  G09B29/10

CPC分类号： G06T11/001 ,  G06F3/011 ,  G06T3/60 ,  G09B29/106 ,  G06T2200/24

摘要： An augmented reality device may communicate with a map server via an API interface to provide mapping data that may be implemented into a canonical map, and may also receive map data from the map server. A visualization of map quality, including quality indicators for multiple cells of the environment, may be provided to the user as an overlay to the current real-world environment seen through the AR device. These visualizations may include, for example, a map quality minimap and/or a map quality overlay. The visualizations provide guidance to the user that allows more efficient updates to the map, thereby improving map quality and localization of users into the map.