公开(公告)号：US11775058B2

公开(公告)日：2023-10-03

申请号：US17129669

申请日：2020-12-21

Applicant: Magic Leap, Inc.

Inventor： Vijay Badrinarayanan ,  Zhengyang Wu ,  Srivignesh Rajendran ,  Andrew Rabinovich

IPC: G06T7/00 ,  G06F3/01 ,  G06T7/11 ,  G06N3/08 ,  G06V40/19 ,  G06V10/764 ,  G06V10/82 ,  G06V40/18

CPC classification number: G06F3/013 ,  G06N3/08 ,  G06T7/0012 ,  G06T7/11 ,  G06V10/764 ,  G06V10/82 ,  G06V40/18 ,  G06V40/19 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30041

Abstract: Systems and methods for estimating a gaze vector of an eye using a trained neural network. An input image of the eye may be received from a camera. The input image may be provided to the neural network. Network output data may be generated using the neural network. The network output data may include two-dimensional (2D) pupil data, eye segmentation data, and/or cornea center data. The gaze vector may be computed based on the network output data. The neural network may be previously trained by providing a training input image to the neural network, generating training network output data, receiving ground-truth (GT) data, computing error data based on a difference between the training network output data and the GT data, and modifying the neural network based on the error data.

公开(公告)号：US20210182554A1

公开(公告)日：2021-06-17

申请号：US17129669

申请日：2020-12-21

Applicant: Magic Leap, Inc.

Inventor： Vijay Badrinarayanan ,  Zhengyang Wu ,  Srivignesh Rajendran ,  Andrew Rabinovich

IPC: G06K9/00 ,  G06T7/11 ,  G06T7/00 ,  G06N3/08

Abstract: Systems and methods for estimating a gaze vector of an eye using a trained neural network. An input image of the eye may be received from a camera. The input image may be provided to the neural network. Network output data may be generated using the neural network. The network output data may include two-dimensional (2D) pupil data, eye segmentation data, and/or cornea center data. The gaze vector may be computed based on the network output data. The neural network may be previously trained by providing a training input image to the neural network, generating training network output data, receiving ground-truth (GT) data, computing error data based on a difference between the training network output data and the GT data, and modifying the neural network based on the error data.

公开(公告)号：US20220244781A1

公开(公告)日：2022-08-04

申请号：US17674724

申请日：2022-02-17

Applicant: Magic Leap, Inc.

Inventor： Zhengyang Wu ,  Srivignesh Rajendran ,  Tarrence van As ,  Joelle Zimmermann ,  Vijay Badrinarayanan ,  Andrew Rabinovich

IPC: G06F3/01 ,  G06V10/26 ,  G06V40/18 ,  G06V10/82 ,  G06V10/774 ,  G02B27/00

Abstract: Techniques related to the computation of gaze vectors of users of wearable devices are disclosed. A neural network may be trained through first and second training steps. The neural network may include a set of feature encoding layers and a plurality of sets of task-specific layers that each operate on an output of the set of feature encoding layers. During the first training step, a first image of a first eye may be provided to the neural network, eye segmentation data may be generated using the neural network, and the set of feature encoding layers may be trained. During the second training step, a second image of a second eye may be provided to the neural network, network output data may be generated using the neural network, and the plurality of sets of task-specific layers may be trained.