公开(公告)号：US11290706B2

公开(公告)日：2022-03-29

申请号：US17125932

申请日：2020-12-17

Applicant: Magic Leap, Inc.

Inventor： Lionel Ernest Edwin ,  Zachary C. Nienstedt ,  Ivan Li Chuen Yeoh ,  Samuel A. Miller ,  Yan Xu ,  Jordan Alexander Cazamias

IPC: H04N13/344 ,  G02B27/01 ,  G02B27/00 ,  G06F3/04815 ,  H04N13/383 ,  G06F3/01 ,  G02B30/40

Abstract: A wearable device may include a head-mounted display (HMD) for rendering a three-dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the HMD. For example, the HMD-to-eye alignment vary for different users and may change over time (e.g., as a given user moves around or as the HMD slips or otherwise becomes displaced). The wearable device may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and may take actions to reduce or minimize effects of any misalignment.

公开(公告)号：US12196952B2

公开(公告)日：2025-01-14

申请号：US18521613

申请日：2023-11-28

Applicant: Magic Leap, Inc.

Inventor： Lionel Ernest Edwin ,  Zachary C. Nienstedt ,  Ivan Li Chuen Yeoh ,  Samuel A. Miller ,  Yan Xu ,  Jordan Alexander Cazamias

IPC: G02B27/00 ,  A61B3/11 ,  A61B3/113 ,  G02B27/01 ,  G02B30/00 ,  G02B30/40 ,  G06F1/16 ,  G06F3/01 ,  G06F3/0346 ,  G06F3/04815 ,  G06F3/16 ,  G06T3/40 ,  G06V10/42 ,  G06V10/46 ,  G06V10/60 ,  G06V40/18 ,  H04N13/344 ,  H04N13/383

Abstract: A wearable device may include a head-mounted display (HMD) for rendering a three-dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the HMD. For example, the HMD-to-eye alignment may vary for different users and may change over time (e.g., as a user moves around and/or the HMD slips or is otherwise displaced). The wearable device may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and may take actions to reduce or minimize effects of any misalignment.

公开(公告)号：US20200026348A1

公开(公告)日：2020-01-23

申请号：US16588830

申请日：2019-09-30

Applicant: Magic Leap, Inc.

Inventor： Zachary C. Nienstedt ,  Samuel A. Miller ,  Barak Freedman ,  Lionel Ernest Edwin ,  Eric C. Browy ,  William Hudson Welch ,  Ron Liraz Lidji

IPC: G06F3/01 ,  G02B27/01 ,  G06F1/16

Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

公开(公告)号：US11880033B2

公开(公告)日：2024-01-23

申请号：US17706429

申请日：2022-03-28

Applicant: Magic Leap, Inc.

Inventor： Lionel Ernest Edwin ,  Zachary C. Nienstedt ,  Ivan Li Chuen Yeoh ,  Samuel A. Miller ,  Yan Xu ,  Jordan Alexander Cazamias

IPC: G02B27/00 ,  H04N13/344 ,  G06F3/04815 ,  G02B27/01 ,  H04N13/383 ,  G06F3/01 ,  G02B30/40 ,  G06V40/18 ,  A61B3/11 ,  A61B3/113 ,  G06F1/16 ,  G06F3/0346 ,  G06F3/16 ,  G02B30/00 ,  G06V10/46 ,  G06V10/42 ,  G06V10/60 ,  G06T3/40

CPC classification number: G02B27/0081 ,  A61B3/111 ,  A61B3/113 ,  G02B27/0093 ,  G02B27/0172 ,  G02B27/0176 ,  G02B30/00 ,  G02B30/40 ,  G06F1/163 ,  G06F3/011 ,  G06F3/012 ,  G06F3/013 ,  G06F3/017 ,  G06F3/0346 ,  G06F3/04815 ,  G06F3/16 ,  G06F3/167 ,  G06V40/193 ,  H04N13/344 ,  H04N13/383 ,  G02B2027/0134 ,  G02B2027/0138 ,  G02B2027/0174 ,  G06T3/40 ,  G06V10/42 ,  G06V10/467 ,  G06V10/60

Abstract: A wearable device may include a head-mounted display (HMD) for rendering a three-dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the HMD. For example, the HMD-to-eye alignment may vary for different users and change over time (e.g., as a given user moves around or as the HMD slips or otherwise becomes displaced). The wearable device may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and may take actions to reduce or minimize effects of any misalignment.

公开(公告)号：US20210105456A1

公开(公告)日：2021-04-08

申请号：US17125932

申请日：2020-12-17

Applicant: Magic Leap, Inc.

Inventor： Lionel Ernest Edwin ,  Zachary C. Nienstedt ,  Ivan Li Chuen Yeoh ,  Samuel A. Miller ,  Yan Xu ,  Jordan Alexander Cazamias

IPC: H04N13/344 ,  G06F3/0481 ,  G02B27/01 ,  H04N13/383 ,  G06F3/01 ,  G02B27/00 ,  G02B30/00 ,  G02B30/40

Abstract: A wearable device may include a head-mounted display (HMD) for rendering a three-dimensional (3D) virtual object which appears to be located in an ambient environment of a user of the display. The relative positions of the HMD and one or more eyes of the user may not be in desired positions to receive, or register, image information outputted by the HMD. For example, the HMD-to-eye alignment vary for different users and may change over time (e.g., as a given user moves around or as the HMD slips or otherwise becomes displaced). The wearable device may determine a relative position or alignment between the HMD and the user's eyes. Based on the relative positions, the wearable device may determine if it is properly fitted to the user, may provide feedback on the quality of the fit to the user, and may take actions to reduce or minimize effects of any misalignment.

公开(公告)号：US11989339B2

公开(公告)日：2024-05-21

申请号：US18112573

申请日：2023-02-22

Applicant: Magic Leap, Inc.

Inventor： Zachary C. Nienstedt ,  Samuel A. Miller ,  Barak Freedman ,  Lionel Ernest Edwin ,  Eric C. Browy ,  William Hudson Welch ,  Ron Liraz Lidji

IPC: G06F3/01 ,  G02B27/01 ,  G06F1/16 ,  G06F3/16

CPC classification number: G06F3/011 ,  G02B27/017 ,  G06F1/1686 ,  G06F1/1694 ,  G02B2027/0187

Abstract: A method of performing localization of a handheld device with respect to a wearable device includes capturing, by a first imaging device mounted to the handheld device, a fiducial image containing a number of fiducials affixed to the wearable device and capturing, by a second imaging device mounted to the handheld device, a world image containing one or more features surrounding the handheld device. The method also includes obtaining, by a sensor mounted to the handheld device, handheld data indicative of movement of the handheld device, determining the number of fiducials contained in the fiducial image, and updating a position and an orientation of the handheld device using at least one of the fiducial image or the world image and the handheld data.

公开(公告)号：US11625090B2

公开(公告)日：2023-04-11

申请号：US17497782

申请日：2021-10-08

Applicant: Magic Leap, Inc.

Inventor： Zachary C. Nienstedt ,  Samuel A. Miller ,  Barak Freedman ,  Lionel Ernest Edwin ,  Eric C. Browy ,  William Hudson Welch ,  Ron Liraz Lidji

IPC: G06F3/01 ,  G02B27/01 ,  G06F1/16

Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

公开(公告)号：US10860090B2

公开(公告)日：2020-12-08

申请号：US16588830

申请日：2019-09-30

Applicant: Magic Leap, Inc.

Inventor： Zachary C. Nienstedt ,  Samuel A. Miller ,  Barak Freedman ,  Lionel Ernest Edwin ,  Eric C. Browy ,  William Hudson Welch ,  Ron Liraz Lidji

IPC: G06F3/01 ,  G02B27/01 ,  G06F1/16

Abstract: Techniques are disclosed for performing localization of a handheld device with respect to a wearable device. At least one sensor mounted to the handheld device, such as an inertial measurement unit (IMU), may obtain handheld data indicative of movement of the handheld device with respect to the world. An imaging device mounted to either the handheld device or the wearable device may capture a fiducial image containing a number of fiducials affixed to the other device. The number of fiducials contained in the image are determined. Based on the number of fiducials, at least one of a position and an orientation of the handheld device with respect to the wearable device are updated based on the image and the handheld data in accordance with a first operating state, a second operating state, or a third operating state.

公开(公告)号：US20240256029A1

公开(公告)日：2024-08-01

申请号：US18632182

申请日：2024-04-10

Applicant: Magic Leap, Inc.

Inventor： Zachary C. Nienstedt ,  Samuel A. Miller ,  Barak Freedman ,  Lionel Ernest Edwin ,  Eric C. Browy ,  William Hudson Welch ,  Ron Liraz Lidji

IPC: G06F3/01 ,  G02B27/01 ,  G06F1/16

CPC classification number: G06F3/011 ,  G02B27/017 ,  G06F1/1686 ,  G06F1/1694 ,  G02B2027/0187

Abstract: An augmented reality (AR) system includes a wearable device comprising a display inside the wearable device and operable to display virtual content and an imaging device mounted to the wearable device. The AR system also includes a handheld device comprising handheld fiducials affixed to the handheld device and a computing apparatus configured to perform localization of the handheld device with respect to the wearable device.

公开(公告)号：US20230195212A1

公开(公告)日：2023-06-22

申请号：US18112573

申请日：2023-02-22

Applicant: Magic Leap, Inc.

Inventor： Zachary C. Nienstedt ,  Samuel A. Miller ,  Barak Freedman ,  Lionel Ernest Edwin ,  Eric C. Browy ,  William Hudson Welch ,  Ron Liraz Lidji

IPC: G06F3/01 ,  G02B27/01 ,  G06F1/16

CPC classification number: G06F3/011 ,  G02B27/017 ,  G06F1/1686 ,  G06F1/1694 ,  G02B2027/0187

Abstract: A method of performing localization of a handheld device with respect to a wearable device includes capturing, by a first imaging device mounted to the handheld device, a fiducial image containing a number of fiducials affixed to the wearable device and capturing, by a second imaging device mounted to the handheld device, a world image containing one or more features surrounding the handheld device. The method also includes obtaining, by a sensor mounted to the handheld device, handheld data indicative of movement of the handheld device, determining the number of fiducials contained in the fiducial image, and updating a position and an orientation of the handheld device using at least one of the fiducial image or the world image and the handheld data.