-
公开(公告)号:US20210182554A1
公开(公告)日:2021-06-17
申请号:US17129669
申请日:2020-12-21
Applicant: Magic Leap, Inc.
Inventor: Vijay Badrinarayanan , Zhengyang Wu , Srivignesh Rajendran , Andrew Rabinovich
Abstract: Systems and methods for estimating a gaze vector of an eye using a trained neural network. An input image of the eye may be received from a camera. The input image may be provided to the neural network. Network output data may be generated using the neural network. The network output data may include two-dimensional (2D) pupil data, eye segmentation data, and/or cornea center data. The gaze vector may be computed based on the network output data. The neural network may be previously trained by providing a training input image to the neural network, generating training network output data, receiving ground-truth (GT) data, computing error data based on a difference between the training network output data and the GT data, and modifying the neural network based on the error data.
-
公开(公告)号:US20200234051A1
公开(公告)日:2020-07-23
申请号:US16844812
申请日:2020-04-09
Applicant: Magic Leap, Inc.
Inventor: Chen-Yu Lee , Vijay Badrinarayanan , Tomasz Jan Malisiewicz , Andrew Rabinovich
Abstract: Systems and methods for estimating a layout of a room are disclosed. The room layout can comprise the location of a floor, one or more walls, and a ceiling. In one aspect, a neural network can analyze an image of a portion of a room to determine the room layout. The neural network can comprise a convolutional neural network having an encoder sub-network, a decoder sub-network, and a side sub-network. The neural network can determine a three-dimensional room layout using two-dimensional ordered keypoints associated with a room type. The room layout can be used in applications such as augmented or mixed reality, robotics, autonomous indoor navigation, etc.
-
公开(公告)号:US10573042B2
公开(公告)日:2020-02-25
申请号:US15717747
申请日:2017-09-27
Applicant: Magic Leap, Inc.
Inventor: Adrian Kaehler , Gary Bradski , Vijay Badrinarayanan
Abstract: A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device fits the user, or whether an adjustment to a rendering location of virtual object should be made to compensate for a deviation of the wearable device from its normal resting position.
-
公开(公告)号:US20180137642A1
公开(公告)日:2018-05-17
申请号:US15812928
申请日:2017-11-14
Applicant: Magic Leap, Inc.
Inventor: Tomasz Malisiewicz , Andrew Rabinovich , Vijay Badrinarayanan , Debidatta Dwibedi
CPC classification number: G06T7/70 , G06K9/00201 , G06K9/4628 , G06K9/6271 , G06K9/66 , G06N3/0445 , G06N3/0454 , G06N3/08 , G06N3/084 , G06T7/11 , G06T2210/12
Abstract: Systems and methods for cuboid detection and keypoint localization in images are disclosed. In one aspect, a deep cuboid detector can be used for simultaneous cuboid detection and keypoint localization in monocular images. The deep cuboid detector can include a plurality of convolutional layers and non-convolutional layers of a trained convolution neural network for determining a convolutional feature map from an input image. A region proposal network of the deep cuboid detector can determine a bounding box surrounding a cuboid in the image using the convolutional feature map. The pooling layer and regressor layers of the deep cuboid detector can implement iterative feature pooling for determining a refined bounding box and a parameterized representation of the cuboid.
-
公开(公告)号:US20180096503A1
公开(公告)日:2018-04-05
申请号:US15717747
申请日:2017-09-27
Applicant: Magic Leap, Inc.
Inventor: Adrian Kaehler , Gary Bradski , Vijay Badrinarayanan
CPC classification number: G06T11/60 , G02B27/0172 , G02B27/0179 , G02B2027/0127 , G02B2027/0134 , G02B2027/0138 , G02B2027/014 , G02B2027/0178 , G02B2027/0181 , G02B2027/0187 , G06K9/00248 , G06K9/00255 , G06K9/00281 , G06K9/00604 , G06K9/0061 , G06K9/00617 , G06K9/4671 , G06T3/20
Abstract: A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device fits the user, or whether an adjustment to a rendering location of virtual object should be made to compensate for a deviation of the wearable device from its normal resting position.
-
公开(公告)号:US20170262737A1
公开(公告)日:2017-09-14
申请号:US15457990
申请日:2017-03-13
Applicant: Magic Leap, Inc.
Inventor: Andrew Rabinovich , Vijay Badrinarayanan , Daniel DeTone , Srivignesh Rajendran , Douglas Bertram Lee , Tomasz Malisiewicz
CPC classification number: G06K9/66 , G06K9/4628 , G06K9/6267 , G06K9/6272
Abstract: The present disclosure provides an improved approach to implement structure learning of neural networks by exploiting correlations in the data/problem the networks aim to solve. A greedy approach is described that finds bottlenecks of information gain from the bottom convolutional layers all the way to the fully connected layers. Rather than simply making the architecture deeper, additional computation and capacitance is only added where it is required.
-
公开(公告)号:US11853894B2
公开(公告)日:2023-12-26
申请号:US17344758
申请日:2021-06-10
Applicant: Magic Leap, Inc.
Inventor: Andrew Rabinovich , Vijay Badrinarayanan , Srivignesh Rajendran , Chen-Yu Lee
CPC classification number: G06N3/084 , G06F18/217 , G06N3/04 , G06N3/044 , G06N3/047
Abstract: Methods and systems for meta-learning are described for automating learning of child tasks with a single neural network. The order in which tasks are learned by the neural network can affect performance of the network, and the meta-learning approach can use a task-level curriculum for multi-task training. The task-level curriculum can be learned by monitoring a trajectory of loss functions during training. The meta-learning approach can learn to adapt task loss balancing weights in the course of training to get improved performance on multiple tasks on real world datasets. Advantageously, learning to dynamically balance weights among different task losses can lead to superior performance over the use of static weights determined by expensive random searches or heuristics. Embodiments of the meta-learning approach can be used for computer vision tasks or natural language processing tasks, and the trained neural networks can be used by augmented or virtual reality devices.
-
公开(公告)号:US11630314B2
公开(公告)日:2023-04-18
申请号:US17719885
申请日:2022-04-13
Applicant: Magic Leap, Inc.
Inventor: Adrian Kaehler , Gary R. Bradski , Vijay Badrinarayanan
IPC: G06F3/01 , G02B27/01 , G06N3/10 , G06V20/10 , G06V20/20 , G06V40/20 , G06F18/214 , G06F18/00 , G06N3/044 , G06N3/045 , G06N3/048 , G06T7/70 , G06F3/0346 , G06F3/04815 , G06N3/08 , G06T19/00 , G06N5/025 , G06V10/44 , G06N3/047 , G06N5/01 , G06N7/01
Abstract: An example wearable display system can be capable of determining a user interface (UI) event with respect to a virtual UI device (e.g., a button) and a pointer (e.g., a finger or a stylus) using a neural network. The wearable display system can render a representation of the UI device onto an image of the pointer captured when the virtual UI device is shown to the user and the user uses the pointer to interact with the virtual UI device. The representation of the UI device can include concentric shapes (or shapes with similar or the same centers of gravity) of high contrast. The neural network can be trained using training images with representations of virtual UI devices and pointers.
-
公开(公告)号:US20220237815A1
公开(公告)日:2022-07-28
申请号:US17717696
申请日:2022-04-11
Applicant: Magic Leap, Inc.
Inventor: Tomasz Jan Malisiewicz , Andrew Rabinovich , Vijay Badrinarayanan , Debidatta Dwibedi
Abstract: Systems and methods for cuboid detection and keypoint localization in images are disclosed. In one aspect, a deep cuboid detector can be used for simultaneous cuboid detection and keypoint localization in monocular images. The deep cuboid detector can include a plurality of convolutional layers and non-convolutional layers of a trained convolution neural network for determining a convolutional feature map from an input image. A region proposal network of the deep cuboid detector can determine a bounding box surrounding a cuboid in the image using the convolutional feature map. The pooling layer and regressor layers of the deep cuboid detector can implement iterative feature pooling for determining a refined bounding box and a parameterized representation of the cuboid.
-
公开(公告)号:US11328443B2
公开(公告)日:2022-05-10
申请号:US17146799
申请日:2021-01-12
Applicant: Magic Leap, Inc.
Inventor: Tomasz Jan Malisiewicz , Andrew Rabinovich , Vijay Badrinarayanan , Debidatta Dwibedi
Abstract: Systems and methods for cuboid detection and keypoint localization in images are disclosed. In one aspect, a deep cuboid detector can be used for simultaneous cuboid detection and keypoint localization in monocular images. The deep cuboid detector can include a plurality of convolutional layers and non-convolutional layers of a trained convolution neural network for determining a convolutional feature map from an input image. A region proposal network of the deep cuboid detector can determine a bounding box surrounding a cuboid in the image using the convolutional feature map. The pooling layer and regressor layers of the deep cuboid detector can implement iterative feature pooling for determining a refined bounding box and a parameterized representation of the cuboid.
-
-
-
-
-
-
-
-
-