公开(公告)号：US20250118010A1

公开(公告)日：2025-04-10

申请号：US18903411

申请日：2024-10-01

Applicant: NEC Laboratories America, Inc.

Inventor： Ziyu Jiang ,  Bingbing Zhuang ,  Manmohan Chandraker

IPC: G06T15/08 ,  G06T15/06 ,  G06T15/50

Abstract: A computer-implemented method for synthesizing an image includes capturing data from a scene and decomposing the captured scene into static objects; dynamic objects and sky. Bounding boxes are generated for the dynamic objects and motion is simulated for the dynamic objects as static movement of the bounding boxes. The dynamic objects and the static objects are merged according to density and color of sample points. The sky is blended into a merged version of the dynamic objects and the static objects, and an image is synthesized from volume rendered rays.

公开(公告)号：US20250115254A1

公开(公告)日：2025-04-10

申请号：US18905738

申请日：2024-10-03

Applicant: NEC Laboratories America, Inc.

Inventor： Buyu Liu ,  Francesco Pittaluga ,  Bingbing Zhuang ,  Manmohan Chandraker ,  Samuel Sohn

IPC: B60W50/00 ,  B60W60/00 ,  G05B13/02

Abstract: Systems and methods for a hybrid motion planner for autonomous vehicles. A multi-lane intelligent driver model (MIDM) can predict trajectory predictions from collected data by considering adjacent lanes of an ego vehicle. A multi-lane hybrid planning driver model (MPDM) can be trained using open-loop ground truth data and close-loop simulations to obtain a trained MPDM. The trained MPDM can predict planned trajectories with collected data and the trajectory predictions to generate final trajectories for the autonomous vehicles. The final trajectories can be employed to control the autonomous vehicles.

公开(公告)号：US12131557B2

公开(公告)日：2024-10-29

申请号：US17521193

申请日：2021-11-08

Applicant: NEC Laboratories America, Inc.

Inventor： Buyu Liu ,  Bingbing Zhuang ,  Manmohan Chandraker

IPC: G06T7/10 ,  B60W10/18 ,  B60W10/20 ,  B60W30/02 ,  B60W30/09 ,  G06F18/214 ,  G06V20/56 ,  G06V30/262

CPC classification number: G06V20/588 ,  B60W10/18 ,  B60W10/20 ,  B60W30/02 ,  B60W30/09 ,  G06F18/2155 ,  G06T7/10 ,  G06V30/274 ,  B60W2420/403 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/30256

Abstract: A computer-implemented method for road layout prediction is provided. The method includes segmenting, by a first processor-based element, an RGB image to output pixel-level semantic segmentation results for the RGB image in a perspective view for both visible and occluded pixels in the perspective view based on contextual clues. The method further includes learning, by a second processor-based element, a mapping from the pixel-level semantic segmentation results for the RGB image in the perspective view to a top view of the RGB image using a road plane assumption. The method also includes generating, by a third processor-based element, an occlusion-aware parametric road layout prediction for road layout related attributes in the top view.

公开(公告)号：US20240355090A1

公开(公告)日：2024-10-24

申请号：US18639534

申请日：2024-04-18

Applicant: NEC Laboratories America, Inc.

Inventor： Vijay Kumar Baikampady Gopalkrishna ,  Samuel Schulter ,  Xiang Yu ,  Manmohan Chandraker

IPC: G06V10/74 ,  G06F16/532 ,  G06F40/186 ,  G06V10/774 ,  G06V20/60 ,  G06V20/70

CPC classification number: G06V10/761 ,  G06F16/532 ,  G06V10/774 ,  G06V20/60 ,  G06V20/70 ,  G06F40/186

Abstract: Systems and methods are provided for matching one or more images using conditional similarity pseudo-labels, including analyzing an unlabeled dataset of images, accessing a foundational vision-language model trained on a plurality of image-text pairs, and defining a set of attributes each comprising multiple possible values for generating pseudo-labels based on notions of similarity (NoS). Text prompts are generated for each attribute value using a prompt template and encoding the text prompts using a text encoder of the foundational model. Each image in the dataset of images is processed through a vision encoder of the foundational model to obtain visual features, the visual features are compared against encoded text prompts to assign a pseudo-label for each attribute for each image, and a conditional similarity network (CSN) is trained with the pseudo-labeled images to generate a conditional similarity model.

公开(公告)号：US20240354642A1

公开(公告)日：2024-10-24

申请号：US18616930

申请日：2024-03-26

Applicant: NEC Laboratories America, Inc.

Inventor： Vijay Kumar Baikampady Gopalkrishna ,  Manmohan Chandraker

IPC: G06N20/00

CPC classification number: G06N20/00

Abstract: Methods and systems for fine-tuning a model include generating a label space for a target domain. Text pseudo-labels are generated for images in an unlabeled dataset from the target domain based on the label space using a pre-trained vision language model. The pre-trained vision language model is fine-tuned for the target domain using the images with the text pseudo-labels.

公开(公告)号：US20240037187A1

公开(公告)日：2024-02-01

申请号：US18484832

申请日：2023-10-11

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Xiang Yu ,  Bingbing Zhuang ,  Manmohan Chandraker ,  Donghyun Kim

IPC: G06F18/213 ,  G06N3/08 ,  G06V10/75 ,  G06F18/22 ,  G06F18/214

CPC classification number: G06F18/213 ,  G06N3/08 ,  G06V10/751 ,  G06F18/22 ,  G06F18/2155

Abstract: Video methods and systems include extracting features of a first modality and a second modality from a labeled first training dataset in a first domain and an unlabeled second training dataset in a second domain. A video analysis model is trained using contrastive learning on the extracted features, including optimization of a loss function that includes a cross-domain regularization part and a cross-modality regularization part.

公开(公告)号：US20240037186A1

公开(公告)日：2024-02-01

申请号：US18484826

申请日：2023-10-11

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Xiang Yu ,  Bingbing Zhuang ,  Manmohan Chandraker ,  Donghyun Kim

IPC: G06F18/213 ,  G06N3/08 ,  G06V10/75 ,  G06F18/22 ,  G06F18/214

CPC classification number: G06F18/213 ,  G06N3/08 ,  G06V10/751 ,  G06F18/22 ,  G06F18/2155

Abstract: Video methods and systems include extracting features of a first modality and a second modality from a labeled first training dataset in a first domain and an unlabeled second training dataset in a second domain. A video analysis model is trained using contrastive learning on the extracted features, including optimization of a loss function that includes a cross-domain regularization part and a cross-modality regularization part.

公开(公告)号：US11816901B2

公开(公告)日：2023-11-14

申请号：US17187157

申请日：2021-02-26

Applicant: NEC Laboratories America, Inc.

Inventor： Sriram Nochur Narayanan ,  Buyu Liu ,  Ramin Moslemi ,  Francesco Pittaluga ,  Manmohan Chandraker

IPC: G06V20/56 ,  B60W60/00 ,  G06N3/08 ,  G06F18/214 ,  B60W30/095 ,  G06V10/82

CPC classification number: G06V20/56 ,  B60W30/0953 ,  B60W30/0956 ,  B60W60/0027 ,  G06F18/214 ,  G06N3/08 ,  G06V10/82 ,  B60W2420/42 ,  B60W2554/4045

Abstract: Methods and systems for training a trajectory prediction model and performing a vehicle maneuver include encoding a set of training data to generate encoded training vectors, where the training data includes trajectory information for agents over time. Trajectory scenarios are simulated based on the encoded training vectors, with each simulated trajectory scenario representing one or more agents with respective agent trajectories, to generate simulated training data. A predictive neural network model is trained using the simulated training data to generate predicted trajectory scenarios based on a detected scene.

公开(公告)号：US11518382B2

公开(公告)日：2022-12-06

申请号：US16696087

申请日：2019-11-26

Applicant: NEC Laboratories America, Inc.

Inventor： Samuel Schulter ,  Nataniel Ruiz ,  Manmohan Chandraker

IPC: B60W30/095 ,  G06N3/08 ,  G06F30/20 ,  G06V20/56 ,  B60W50/00

Abstract: A method is provided for danger prediction. The method includes generating fully-annotated simulated training data for a machine learning model responsive to receiving a set of computer-selected simulator-adjusting parameters. The method further includes training the machine learning model using reinforcement learning on the fully-annotated simulated training data. The method also includes measuring an accuracy of the trained machine learning model relative to learning a discriminative function for a given task. The discriminative function predicts a given label for a given image from the fully-annotated simulated training data. The method additionally includes adjusting the computer-selected simulator-adjusting parameters and repeating said training and measuring steps responsive to the accuracy being below a threshold accuracy. The method further includes predicting a dangerous condition relative to a motor vehicle and providing a warning to an entity regarding the dangerous condition by applying the trained machine learning model to actual unlabeled data for the vehicle.

公开(公告)号：US11455813B2

公开(公告)日：2022-09-27

申请号：US17096111

申请日：2020-11-12

Applicant: NEC Laboratories America, Inc.

Inventor： Buyu Liu ,  Bingbing Zhuang ,  Samuel Schulter ,  Manmohan Chandraker

IPC: G06V30/422 ,  G06T7/00 ,  G06V40/12

Abstract: Systems and methods are provided for producing a road layout model. The method includes capturing digital images having a perspective view, converting each of the digital images into top-down images, and conveying a top-down image of time t to a neural network that performs a feature transform to form a feature map of time t. The method also includes transferring the feature map of the top-down image of time t to a feature transform module to warp the feature map to a time t+1, and conveying a top-down image of time t+1 to form a feature map of time t+1. The method also includes combining the warped feature map of time t with the feature map of time t+1 to form a combined feature map, transferring the combined feature map to a long short-term memory (LSTM) module to generate the road layout model, and displaying the road layout model.