公开(公告)号：WO2020252371A1

公开(公告)日：2020-12-17

申请号：PCT/US2020/037573

申请日：2020-06-12

Applicant: MAGIC LEAP, INC. ,  CHOUDHARY, Siddharth ,  RAMNATH, Divya ,  DONG, Shiyu ,  MAHENDRAN, Siddarth ,  KANNAN, Arumugam Kalai ,  SINGHAL, Prateek ,  GUPTA, Khushi ,  SEKHAR, Nitesh ,  GANGWAR, Manushree

Inventor： CHOUDHARY, Siddharth ,  RAMNATH, Divya ,  DONG, Shiyu ,  MAHENDRAN, Siddarth ,  KANNAN, Arumugam Kalai ,  SINGHAL, Prateek ,  GUPTA, Khushi ,  SEKHAR, Nitesh ,  GANGWAR, Manushree

IPC: G06K9/46 ,  G06K9/36 ,  G06K9/60

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for scalable three-dimensional (3-D) object recognition in a cross reality system. One of the methods includes maintaining object data specifying objects that have been recognized in a scene. A stream of input images of the scene is received, including a stream of color images and a stream of depth images. A color image is provided as input to an object recognition system. A recognition output that identifies a respective object mask for each object in the color image is received. A synchronization system determines a corresponding depth image for the color image. A 3-D bounding box generation system determines a respective 3-D bounding box for each object that has been recognized in the color image. Data specifying one or more 3-D bounding boxes is received as output from the 3-D bounding box generation system.

公开(公告)号：WO2021263035A1

公开(公告)日：2021-12-30

申请号：PCT/US2021/038971

申请日：2021-06-24

Applicant: MAGIC LEAP, INC.

Inventor： MAHENDRAN, Siddharth ,  BANSAL, Nitin ,  SEKHAR, Nitesh ,  GANGWAR, Manushree ,  GUPTA, Khushi ,  SINGHAL, Prateek

IPC: G06K9/62 ,  G06N3/08 ,  G06K9/46 ,  G06T7/00 ,  G06T19/00 ,  G06K9/6261 ,  G06K9/6267 ,  G06N3/04 ,  G06T19/006 ,  G06T2207/20084 ,  G06T7/60 ,  G06T7/73

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for object recognition neural network for amodal center prediction. One of the methods includes receiving an image of an object captured by a camera. The image of the object is processed using an object recognition neural network that is configured to generate an object recognition output. The object recognition output includes data defining a predicted two-dimensional amodal center of the object, wherein the predicted two-dimensional amodal center of the object is a projection of a predicted three-dimensional center of the object under a camera pose of the camera that captured the image.

公开(公告)号：WO2022026603A1

公开(公告)日：2022-02-03

申请号：PCT/US2021/043543

申请日：2021-07-28

Applicant: MAGIC LEAP, INC.

Inventor： MAHENDRAN, Siddharth ,  BANSAL, Nitin ,  SEKHAR, Nitesh ,  GANGWAR, Manushree ,  GUPTA, Khushi ,  SINGHAL, Prateek ,  VAN AS, Tarrence ,  RAO, Adithya Shricharan Srinivasa

IPC: G06N20/00 ,  G06K9/00 ,  G06K9/32 ,  G06T19/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an object recognition neural network using multiple data sources. One of the methods includes receiving training data that includes a plurality of training images from a first source and images from a second source. A set of training images are obtained from the training data. For each training image in the set of training images, contrast equalization is applied to the training image to generate a modified image. The modified image is processed using the neural network to generate an object recognition output for the modified image. A loss is determined based on errors between, for each training image in the set, the object recognition output for the modified image generated from the training image and ground-truth annotation for the training image. Parameters of the neural network are updated based on the determined loss.