公开(公告)号：US20240037186A1

公开(公告)日：2024-02-01

申请号：US18484826

申请日：2023-10-11

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Xiang Yu ,  Bingbing Zhuang ,  Manmohan Chandraker ,  Donghyun Kim

IPC: G06F18/213 ,  G06N3/08 ,  G06V10/75 ,  G06F18/22 ,  G06F18/214

CPC classification number: G06F18/213 ,  G06N3/08 ,  G06V10/751 ,  G06F18/22 ,  G06F18/2155

Abstract: Video methods and systems include extracting features of a first modality and a second modality from a labeled first training dataset in a first domain and an unlabeled second training dataset in a second domain. A video analysis model is trained using contrastive learning on the extracted features, including optimization of a loss function that includes a cross-domain regularization part and a cross-modality regularization part.

公开(公告)号：US11816901B2

公开(公告)日：2023-11-14

申请号：US17187157

申请日：2021-02-26

Applicant: NEC Laboratories America, Inc.

Inventor： Sriram Nochur Narayanan ,  Buyu Liu ,  Ramin Moslemi ,  Francesco Pittaluga ,  Manmohan Chandraker

IPC: G06V20/56 ,  B60W60/00 ,  G06N3/08 ,  G06F18/214 ,  B60W30/095 ,  G06V10/82

CPC classification number: G06V20/56 ,  B60W30/0953 ,  B60W30/0956 ,  B60W60/0027 ,  G06F18/214 ,  G06N3/08 ,  G06V10/82 ,  B60W2420/42 ,  B60W2554/4045

Abstract: Methods and systems for training a trajectory prediction model and performing a vehicle maneuver include encoding a set of training data to generate encoded training vectors, where the training data includes trajectory information for agents over time. Trajectory scenarios are simulated based on the encoded training vectors, with each simulated trajectory scenario representing one or more agents with respective agent trajectories, to generate simulated training data. A predictive neural network model is trained using the simulated training data to generate predicted trajectory scenarios based on a detected scene.

公开(公告)号：US11518382B2

公开(公告)日：2022-12-06

申请号：US16696087

申请日：2019-11-26

Applicant: NEC Laboratories America, Inc.

Inventor： Samuel Schulter ,  Nataniel Ruiz ,  Manmohan Chandraker

IPC: B60W30/095 ,  G06N3/08 ,  G06F30/20 ,  G06V20/56 ,  B60W50/00

Abstract: A method is provided for danger prediction. The method includes generating fully-annotated simulated training data for a machine learning model responsive to receiving a set of computer-selected simulator-adjusting parameters. The method further includes training the machine learning model using reinforcement learning on the fully-annotated simulated training data. The method also includes measuring an accuracy of the trained machine learning model relative to learning a discriminative function for a given task. The discriminative function predicts a given label for a given image from the fully-annotated simulated training data. The method additionally includes adjusting the computer-selected simulator-adjusting parameters and repeating said training and measuring steps responsive to the accuracy being below a threshold accuracy. The method further includes predicting a dangerous condition relative to a motor vehicle and providing a warning to an entity regarding the dangerous condition by applying the trained machine learning model to actual unlabeled data for the vehicle.

公开(公告)号：US11455813B2

公开(公告)日：2022-09-27

申请号：US17096111

申请日：2020-11-12

Applicant: NEC Laboratories America, Inc.

Inventor： Buyu Liu ,  Bingbing Zhuang ,  Samuel Schulter ,  Manmohan Chandraker

IPC: G06V30/422 ,  G06T7/00 ,  G06V40/12

Abstract: Systems and methods are provided for producing a road layout model. The method includes capturing digital images having a perspective view, converting each of the digital images into top-down images, and conveying a top-down image of time t to a neural network that performs a feature transform to form a feature map of time t. The method also includes transferring the feature map of the top-down image of time t to a feature transform module to warp the feature map to a time t+1, and conveying a top-down image of time t+1 to form a feature map of time t+1. The method also includes combining the warped feature map of time t with the feature map of time t+1 to form a combined feature map, transferring the combined feature map to a long short-term memory (LSTM) module to generate the road layout model, and displaying the road layout model.

公开(公告)号：US20220148220A1

公开(公告)日：2022-05-12

申请号：US17519894

申请日：2021-11-05

Applicant: NEC Laboratories America, Inc.

Inventor： Bingbing Zhuang ,  Manmohan Chandraker

IPC: G06T7/73 ,  G06T7/77 ,  G06T1/00

Abstract: A computer-implemented method for fusing geometrical and Convolutional Neural Network (CNN) relative camera pose is provided. The method includes receiving two images having different camera poses. The method further includes inputting the two images into a geometric solver branch to return, as a first solution, an estimated camera pose and an associated pose uncertainty value determined from a Jacobian of a reproduction error function. The method also includes inputting the two images into a CNN branch to return, as a second solution, a predicted camera pose and an associated pose uncertainty value. The method additionally includes fusing, by a processor device, the first solution and the second solution in a probabilistic manner using Bayes' rule to obtain a fused pose.

公开(公告)号：US20220148189A1

公开(公告)日：2022-05-12

申请号：US17520207

申请日：2021-11-05

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Masoud Faraki ,  Yumin Suh ,  Sparsh Garg ,  Manmohan Chandraker ,  Dongwan Kim

IPC: G06T7/11 ,  G06K9/62

Abstract: Methods and systems for training a model include combining data from multiple datasets, the datasets having different respective label spaces. Relationships between labels in the different label spaces are identified. A unified neural network model is trained, using the combined data and the identified relationships to generate a unified model, with a class relational binary cross-entropy loss.

公开(公告)号：US20220147746A1

公开(公告)日：2022-05-12

申请号：US17521193

申请日：2021-11-08

Applicant: NEC Laboratories America, Inc.

Inventor： Buyu Liu ,  Bingbing Zhuang ,  Manmohan Chandraker

IPC: G06K9/00 ,  G06T7/10 ,  G06K9/72 ,  G06K9/62 ,  B60W30/09 ,  B60W10/18 ,  B60W10/20 ,  B60W30/02

Abstract: A computer-implemented method for road layout prediction is provided. The method includes segmenting, by a first processor-based element, an RGB image to output pixel-level semantic segmentation results for the RGB image in a perspective view for both visible and occluded pixels in the perspective view based on contextual clues. The method further includes learning, by a second processor-based element, a mapping from the pixel-level semantic segmentation results for the RGB image in the perspective view to a top view of the RGB image using a road plane assumption. The method also includes generating, by a third processor-based element, an occlusion-aware parametric road layout prediction for road layout related attributes in the top view.

公开(公告)号：US20220147735A1

公开(公告)日：2022-05-12

申请号：US17519986

申请日：2021-11-05

Applicant: NEC Laboratories America, Inc.

Inventor： Yumin Suh ,  Xiang Yu ,  Yi-Hsuan Tsai ,  Masoud Faraki ,  Manmohan Chandraker

IPC: G06K9/00 ,  G06K9/62

Abstract: A method for employing facial information in unsupervised person re-identification is presented. The method includes extracting, by a body feature extractor, body features from a first data stream, extracting, by a head feature extractor, head features from a second data stream, outputting a body descriptor vector from the body feature extractor, outputting a head descriptor vector from the head feature extractor, and concatenating the body descriptor vector and the head descriptor vector to enable a model to generate a descriptor vector.

公开(公告)号：US11321853B2

公开(公告)日：2022-05-03

申请号：US16939604

申请日：2020-07-27

Applicant: NEC Laboratories America, Inc.

Inventor： Pan Ji ,  Quoc-Huy Tran ,  Manmohan Chandraker ,  Yuliang Zou

IPC: G06T7/00 ,  G06T7/246 ,  G06T7/579

Abstract: A computer-implemented method for implementing a self-supervised visual odometry framework using long-term modeling includes, within a pose network of the self-supervised visual odometry framework including a plurality of pose encoders, a convolution long short-term memory (ConvLSTM) module having a first-layer ConvLSTM and a second-layer ConvLSTM, and a pose prediction layer, performing a first stage of training over a first image sequence using photometric loss, depth smoothness loss and pose cycle consistency loss, and performing a second stage of training to finetune the second-layer ConvLSTM over a second image sequence longer than the first image sequence.

公开(公告)号：US11222238B2

公开(公告)日：2022-01-11

申请号：US17094261

申请日：2020-11-10

Applicant: NEC Laboratories America, Inc.

Inventor： Samuel Schulter ,  Gaurav Sharma ,  Yi-Hsuan Tsai ,  Manmohan Chandraker ,  Xiangyun Zhao

IPC: G06K9/62 ,  G06K9/00 ,  G06N3/04 ,  G06K9/20

Abstract: Methods and systems for object detection include training dataset-specific object detectors using respective annotated datasets, each of the annotated datasets including annotations for a respective set of one or more object classes. The annotated datasets are cross-annotated using the dataset-specific object detectors. A unified object detector is trained, using the cross-annotated datasets, to detect all of the object classes of the annotated datasets. Objects are detected in an input image using the unified object detector.