-
公开(公告)号:US11100644B2
公开(公告)日:2021-08-24
申请号:US16570418
申请日:2019-09-13
申请人: Magic Leap, Inc.
IPC分类号: G06T7/00 , G06T7/10 , G06T7/11 , G06T7/194 , G06K9/46 , G06T7/12 , G06K9/62 , G06K9/00 , G06K9/03
摘要: Systems and methods for eye image segmentation and image quality estimation are disclosed. In one aspect, after receiving an eye image, a device such as an augmented reality device can process the eye image using a convolutional neural network with a merged architecture to generate both a segmented eye image and a quality estimation of the eye image. The segmented eye image can include a background region, a sclera region, an iris region, or a pupil region. In another aspect, a convolutional neural network with a merged architecture can be trained for eye image segmentation and image quality estimation. In yet another aspect, the device can use the segmented eye image to determine eye contours such as a pupil contour and an iris contour. The device can use the eye contours to create a polar image of the iris region for computing an iris code or biometric authentication.
-
公开(公告)号:US20210182636A1
公开(公告)日:2021-06-17
申请号:US17183021
申请日:2021-02-23
申请人: MAGIC LEAP, INC.
发明人: Andrew Rabinovich , Vijay Badrinarayanan , Daniel DeTone , Srivignesh Rajendran , Douglas Bertram Lee , Tomasz Malisiewicz
摘要: The present disclosure provides an improved approach to implement structure learning of neural networks by exploiting correlations in the data/problem the networks aim to solve. A greedy approach is described that finds bottlenecks of information gain from the bottom convolutional layers all the way to the fully connected layers. Rather than simply making the architecture deeper, additional computation and capacitance is only added where it is required.
-
公开(公告)号:US20210134000A1
公开(公告)日:2021-05-06
申请号:US17146799
申请日:2021-01-12
申请人: Magic Leap, Inc.
摘要: Systems and methods for cuboid detection and keypoint localization in images are disclosed. In one aspect, a deep cuboid detector can be used for simultaneous cuboid detection and keypoint localization in monocular images. The deep cuboid detector can include a plurality of convolutional layers and non-convolutional layers of a trained convolution neural network for determining a convolutional feature map from an input image. A region proposal network of the deep cuboid detector can determine a bounding box surrounding a cuboid in the image using the convolutional feature map. The pooling layer and regressor layers of the deep cuboid detector can implement iterative feature pooling for determining a refined bounding box and a parameterized representation of the cuboid.
-
公开(公告)号:US10977820B2
公开(公告)日:2021-04-13
申请号:US16880752
申请日:2020-05-21
申请人: Magic Leap, Inc.
IPC分类号: G06T19/00 , G06T7/70 , G06T7/20 , G06F3/01 , G06F3/0481 , G06N3/02 , G02B27/01 , G06F3/0346 , G06T7/246 , G06F1/16 , G02B27/00
摘要: Disclosed herein is a wearable display system for capturing retraining eye images of an eye of a user for retraining a neural network for eye tracking. The system captures retraining eye images using an image capture device when user interface (UI) events occur with respect to UI devices displayed at display locations of a display. The system can generate a retraining set comprising the retraining eye images and eye poses of the eye of the user in the retraining eye images (e.g., related to the display locations of the UI devices) and obtain a retrained neural network that is retrained using the retraining set.
-
公开(公告)号:US10937188B2
公开(公告)日:2021-03-02
申请号:US16810584
申请日:2020-03-05
申请人: Magic Leap, Inc.
摘要: Systems and methods for cuboid detection and keypoint localization in images are disclosed. In one aspect, a deep cuboid detector can be used for simultaneous cuboid detection and keypoint localization in monocular images. The deep cuboid detector can include a plurality of convolutional layers and non-convolutional layers of a trained convolution neural network for determining a convolutional feature map from an input image. A region proposal network of the deep cuboid detector can determine a bounding box surrounding a cuboid in the image using the convolutional feature map. The pooling layer and regressor layers of the deep cuboid detector can implement iterative feature pooling for determining a refined bounding box and a parameterized representation of the cuboid.
-
公开(公告)号:US20200286251A1
公开(公告)日:2020-09-10
申请号:US16880752
申请日:2020-05-21
申请人: Magic Leap, Inc.
IPC分类号: G06T7/70 , G06T7/20 , G06F3/01 , G06F3/0481 , G06N3/02 , G02B27/01 , G06F3/0346 , G06T7/246 , G06F1/16 , G02B27/00
摘要: Disclosed herein is a wearable display system for capturing retraining eye images of an eye of a user for retraining a neural network for eye tracking. The system captures retraining eye images using an image capture device when user interface (UI) events occur with respect to UI devices displayed at display locations of a display. The system can generate a retraining set comprising the retraining eye images and eye poses of the eye of the user in the retraining eye images (e.g., related to the display locations of the UI devices) and obtain a retrained neural network that is retrained using the retraining set.
-
公开(公告)号:US20240330691A1
公开(公告)日:2024-10-03
申请号:US18661377
申请日:2024-05-10
申请人: Magic Leap, Inc.
摘要: Systems and methods for gradient adversarial training of a neural network are disclosed. In one aspect of gradient adversarial training, an auxiliary neural network can be trained to classify a gradient tensor that is evaluated during backpropagation in a main neural network that provides a desired task output. The main neural network can serve as an adversary to the auxiliary network in addition to a standard task-based training procedure. The auxiliary neural network can pass an adversarial gradient signal back to the main neural network, which can use this signal to regularize the weight tensors in the main neural network. Gradient adversarial training of the neural network can provide improved gradient tensors in the main network. Gradient adversarial techniques can be used to train multitask networks, knowledge distillation networks, and adversarial defense networks.
-
公开(公告)号:US11776131B2
公开(公告)日:2023-10-03
申请号:US17407763
申请日:2021-08-20
申请人: Magic Leap, Inc.
IPC分类号: G06T7/00 , G06T7/12 , G06T7/11 , G06T7/194 , G06V10/56 , G06V10/44 , G06V10/98 , G06V40/18 , G06F18/2413 , G06V10/764 , G06V10/82 , G06T7/10
CPC分类号: G06T7/12 , G06F18/2413 , G06T7/0002 , G06T7/10 , G06T7/11 , G06T7/194 , G06V10/454 , G06V10/56 , G06V10/764 , G06V10/82 , G06V10/993 , G06V40/193 , G06V40/197 , G06T2207/10024 , G06T2207/20076 , G06T2207/20081 , G06T2207/20084 , G06T2207/30041 , G06T2207/30168 , G06T2207/30196
摘要: Systems and methods for eye image segmentation and image quality estimation are disclosed. In one aspect, after receiving an eye image, a device such as an augmented reality device can process the eye image using a convolutional neural network with a merged architecture to generate both a segmented eye image and a quality estimation of the eye image. The segmented eye image can include a background region, a sclera region, an iris region, or a pupil region. In another aspect, a convolutional neural network with a merged architecture can be trained for eye image segmentation and image quality estimation. In yet another aspect, the device can use the segmented eye image to determine eye contours such as a pupil contour and an iris contour. The device can use the eye contours to create a polar image of the iris region for computing an iris code or biometric authentication.
-
29.
公开(公告)号:US11537895B2
公开(公告)日:2022-12-27
申请号:US16169840
申请日:2018-10-24
申请人: Magic Leap, Inc.
摘要: Systems and methods for training a multitask network is disclosed. In one aspect, training the multitask network includes determining a gradient norm of a single-task loss adjusted by a task weight for each task, with respect to network weights of the multitask network, and a relative training rate for the task based on the single-task loss for the task. Subsequently, a gradient loss function, comprising a difference between (1) the determined gradient norm for each task and (2) a corresponding target gradient norm, can be determined. An updated task weight for the task can be determined and used in the next iteration of training the multitask network, using a gradient of the gradient loss function with respect to the task weight for the task.
-
公开(公告)号:US20220245404A1
公开(公告)日:2022-08-04
申请号:US17719885
申请日:2022-04-13
申请人: Magic Leap, Inc.
IPC分类号: G06K9/62 , G06N3/04 , G06N3/10 , G06K9/00 , G06F3/01 , G06V20/10 , G06V20/20 , G06V40/20 , G06T7/70 , G06F3/0346 , G06F3/04815 , G06N3/08 , G06T19/00
摘要: Disclosed herein are examples of a wearable display system capable of determining a user interface (UI) event with respect to a virtual UI device (e.g., a button) and a pointer (e.g., a finger or a stylus) using a neural network. The wearable display system can render a representation of the UI device onto an image of the pointer captured when the virtual UI device is shown to the user and the user uses the pointer to interact with the virtual UI device. The representation of the UI device can include concentric shapes (or shapes with similar or the same centers of gravity) of high contrast. The neural network can be trained using training images with representations of virtual UI devices and pointers.
-
-
-
-
-
-
-
-
-