Neural network for eye image segmentation and image quality estimation

    公开(公告)号:US11100644B2

    公开(公告)日:2021-08-24

    申请号:US16570418

    申请日:2019-09-13

    申请人: Magic Leap, Inc.

    摘要: Systems and methods for eye image segmentation and image quality estimation are disclosed. In one aspect, after receiving an eye image, a device such as an augmented reality device can process the eye image using a convolutional neural network with a merged architecture to generate both a segmented eye image and a quality estimation of the eye image. The segmented eye image can include a background region, a sclera region, an iris region, or a pupil region. In another aspect, a convolutional neural network with a merged architecture can be trained for eye image segmentation and image quality estimation. In yet another aspect, the device can use the segmented eye image to determine eye contours such as a pupil contour and an iris contour. The device can use the eye contours to create a polar image of the iris region for computing an iris code or biometric authentication.

    DEEP LEARNING SYSTEM FOR CUBOID DETECTION

    公开(公告)号:US20210134000A1

    公开(公告)日:2021-05-06

    申请号:US17146799

    申请日:2021-01-12

    申请人: Magic Leap, Inc.

    摘要: Systems and methods for cuboid detection and keypoint localization in images are disclosed. In one aspect, a deep cuboid detector can be used for simultaneous cuboid detection and keypoint localization in monocular images. The deep cuboid detector can include a plurality of convolutional layers and non-convolutional layers of a trained convolution neural network for determining a convolutional feature map from an input image. A region proposal network of the deep cuboid detector can determine a bounding box surrounding a cuboid in the image using the convolutional feature map. The pooling layer and regressor layers of the deep cuboid detector can implement iterative feature pooling for determining a refined bounding box and a parameterized representation of the cuboid.

    Deep learning system for cuboid detection

    公开(公告)号:US10937188B2

    公开(公告)日:2021-03-02

    申请号:US16810584

    申请日:2020-03-05

    申请人: Magic Leap, Inc.

    摘要: Systems and methods for cuboid detection and keypoint localization in images are disclosed. In one aspect, a deep cuboid detector can be used for simultaneous cuboid detection and keypoint localization in monocular images. The deep cuboid detector can include a plurality of convolutional layers and non-convolutional layers of a trained convolution neural network for determining a convolutional feature map from an input image. A region proposal network of the deep cuboid detector can determine a bounding box surrounding a cuboid in the image using the convolutional feature map. The pooling layer and regressor layers of the deep cuboid detector can implement iterative feature pooling for determining a refined bounding box and a parameterized representation of the cuboid.

    GRADIENT ADVERSARIAL TRAINING OF NEURAL NETWORKS

    公开(公告)号:US20240330691A1

    公开(公告)日:2024-10-03

    申请号:US18661377

    申请日:2024-05-10

    申请人: Magic Leap, Inc.

    IPC分类号: G06N3/088 G06N3/045 G06N3/084

    CPC分类号: G06N3/088 G06N3/045 G06N3/084

    摘要: Systems and methods for gradient adversarial training of a neural network are disclosed. In one aspect of gradient adversarial training, an auxiliary neural network can be trained to classify a gradient tensor that is evaluated during backpropagation in a main neural network that provides a desired task output. The main neural network can serve as an adversary to the auxiliary network in addition to a standard task-based training procedure. The auxiliary neural network can pass an adversarial gradient signal back to the main neural network, which can use this signal to regularize the weight tensors in the main neural network. Gradient adversarial training of the neural network can provide improved gradient tensors in the main network. Gradient adversarial techniques can be used to train multitask networks, knowledge distillation networks, and adversarial defense networks.

    Gradient normalization systems and methods for adaptive loss balancing in deep multitask networks

    公开(公告)号:US11537895B2

    公开(公告)日:2022-12-27

    申请号:US16169840

    申请日:2018-10-24

    申请人: Magic Leap, Inc.

    IPC分类号: G06N3/08 G06N3/04 G06N20/00

    摘要: Systems and methods for training a multitask network is disclosed. In one aspect, training the multitask network includes determining a gradient norm of a single-task loss adjusted by a task weight for each task, with respect to network weights of the multitask network, and a relative training rate for the task based on the single-task loss for the task. Subsequently, a gradient loss function, comprising a difference between (1) the determined gradient norm for each task and (2) a corresponding target gradient norm, can be determined. An updated task weight for the task can be determined and used in the next iteration of training the multitask network, using a gradient of the gradient loss function with respect to the task weight for the task.