公开(公告)号：US20240354583A1

公开(公告)日：2024-10-24

申请号：US18615535

申请日：2024-03-25

Applicant: NEC Laboratories America, Inc.

Inventor： Sparsh Garg ,  Samuel Schulter ,  Bingbing Zhuang ,  Manmohan Chandraker

IPC: G06N3/0895

CPC classification number: G06N3/0895

Abstract: Methods and systems for training a model include annotating a subset of an unlabeled training dataset, that includes images of road scenes, with labels. A road defect detection model is iteratively trained, including adding pseudo-labels to a remainder of examples from the unlabeled training dataset and training the road defect detection model based on the labels and the pseudo-labels.

公开(公告)号：US20230081913A1

公开(公告)日：2023-03-16

申请号：US17903393

申请日：2022-09-06

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Bingbing Zhuang ,  Samuel Schulter ,  Buyu Liu ,  Sparsh Garg ,  Ramin Moslemi ,  Inkyu Shin

IPC: G06V10/80 ,  G01S17/89 ,  G06V10/776

Abstract: Systems and methods are provided for multi-modal test-time adaptation. The method includes inputting a digital image into a pre-trained Camera Intra-modal Pseudo-label Generator, and inputting a point cloud set into a pre-trained Lidar Intra-modal Pseudo-label Generator. The method further includes applying a fast 2-dimension (2D) model, and a slow 2D model, to the inputted digital image to apply pseudo-labels, and applying a fast 3-dimension (3D) model, and a slow 3D model, to the inputted point cloud set to apply pseudo-labels. The method further includes fusing pseudo-label predictions from the fast models and the slow models through an Inter-modal Pseudo-label Refinement module to obtain robust pseudo labels, and measuring a prediction consistency for the pseudo-labels. The method further includes selecting confident pseudo-labels from the robust pseudo labels and measured prediction consistencies to form a final cross-modal pseudo-label set as a self-training signal, and updating batch parameters utilizing the self-training signal.

公开(公告)号：US20240355102A1

公开(公告)日：2024-10-24

申请号：US18609097

申请日：2024-03-19

Applicant: NEC Laboratories America, Inc.

Inventor： Sparsh Garg ,  Samuel Schulter

IPC: G06V10/774 ,  G06V10/26 ,  G06V10/82 ,  G06V20/54 ,  G06V20/62

CPC classification number: G06V10/7753 ,  G06V10/26 ,  G06V10/82 ,  G06V20/54 ,  G06V20/625 ,  G06V2201/08

Abstract: Systems and methods for traffic violation prediction. The systems and methods include obtaining a plurality of bounding boxes of road scene categories from an input dataset by employing a pre-trained detection model. A plurality of pseudo-labels of road scene categories for the plurality of bounding boxes can be obtained by employing the pre-trained detection model. A labeled dataset can be obtained by filtering the input dataset for images having the plurality of pseudo-labels and the plurality of bounding boxes. A traffic violation prediction model can be trained with both unlabeled and labeled dataset including the road scene categories obtained from the pre-trained detection model to predict simultaneous traffic violations of one or more riders in a road scene.

公开(公告)号：US20220148189A1

公开(公告)日：2022-05-12

申请号：US17520207

申请日：2021-11-05

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Masoud Faraki ,  Yumin Suh ,  Sparsh Garg ,  Manmohan Chandraker ,  Dongwan Kim

IPC: G06T7/11 ,  G06K9/62

Abstract: Methods and systems for training a model include combining data from multiple datasets, the datasets having different respective label spaces. Relationships between labels in the different label spaces are identified. A unified neural network model is trained, using the combined data and the identified relationships to generate a unified model, with a class relational binary cross-entropy loss.

公开(公告)号：US20250148757A1

公开(公告)日：2025-05-08

申请号：US18931681

申请日：2024-10-30

Applicant: NEC Laboratories America, Inc.

Inventor： Jong-Chyi Su ,  Sparsh Garg ,  Samuel Schulter ,  Manmohan Chandraker ,  Mingfu Liang

IPC: G06V10/764 ,  B60W60/00 ,  G06T11/00 ,  G06V10/25 ,  G06V10/44 ,  G06V10/74 ,  G06V20/70

Abstract: Systems and methods for a self-improving data engine for autonomous vehicles is presented. To train the self-improving data engine for autonomous vehicles (SIDE), multi-modality dense captioning (MMDC) models can detect unrecognized classes from diversified descriptions for input images. A vision-language-model (VLM) can generate textual features from the diversified descriptions and image features from corresponding images to the diversified descriptions. Curated features, including curated textual features and curated image features, can be obtained by comparing similarity scores between the textual features and top-ranked image features based on their likelihood scores. Generate annotations, including bounding boxes and labels, can be generated for the curated features by comparing the similarity scores of labels generated by a zero-shot classifier and the curated textual features. The SIDE can be trained using the curated features, annotations, and feedback.

公开(公告)号：US20240071105A1

公开(公告)日：2024-02-29

申请号：US18453439

申请日：2023-08-22

Applicant: NEC Laboratories America, Inc.

Inventor： Samuel Schulter ,  Bingbing Zhuang ,  Vijay Kumar Baikampady Gopalkrishna ,  Sparsh Garg ,  Zhixing Zhang

IPC: G06V20/56 ,  G06V10/774 ,  G06V10/80 ,  G06V10/82

CPC classification number: G06V20/588 ,  G06V10/7753 ,  G06V10/811 ,  G06V10/82 ,  G06V2201/07

Abstract: Methods and systems for training a model include pre-training a backbone model with a pre-training decoder, using an unlabeled dataset with multiple distinct sensor data modalities that derive from different sensor types. The backbone model is fine-tuned with an output decoder after pre-training, using a labeled dataset with the multiple modalities.

公开(公告)号：US20230281977A1

公开(公告)日：2023-09-07

申请号：US18188766

申请日：2023-03-23

Applicant: NEC Laboratories America, Inc.

Inventor： Samuel Schulter ,  Sparsh Garg ,  Manmohan Chandraker

IPC: G06V10/776 ,  G06T7/11 ,  G06V20/70 ,  H04N17/00 ,  G06V10/774 ,  G06V10/74 ,  G06T7/00

CPC classification number: G06V10/776 ,  G06T7/11 ,  G06V20/70 ,  H04N17/002 ,  G06V10/774 ,  G06V10/761 ,  G06T7/0002 ,  G06T2207/20081 ,  G06T2207/20021 ,  G06T2207/20084

Abstract: Methods and systems for detecting faults include capturing an image of a scene using a camera. The image is embedded using a segmentation model that includes an image branch having an image embedding layer that embeds images into a joint latent space and a text branch having a text embedding layer that embeds text into the joint latent space. Semantic information is generated for a region of the image corresponding to a predetermined static object using the embedded image. A fault of the camera is identified based on a discrepancy between the semantic information and semantic information of the predetermined static image. The fault of the camera is corrected.

公开(公告)号：US20230088335A1

公开(公告)日：2023-03-23

申请号：US17941676

申请日：2022-09-09

Applicant: NEC Laboratories America, Inc.

Inventor： Sparsh Garg ,  Samuel Schulter ,  Vijay Kumar Baikampady Gopalkrishna

IPC: G06V10/774 ,  G06T7/00 ,  G06V20/58 ,  G06V20/56 ,  G06V10/82 ,  G06V20/70

Abstract: Systems and methods is provided for road hazard analysis. The method includes obtaining sensor data of a road environment including a road and observable surroundings, and applying labels to the sensor data. The method further includes training a first neural network model to identify road hazards, training a second neural network model to identify faded lane markings, and training a third neural network model to identify overhanging trees and blocking foliage. The method further includes implementing the trained neural network models to detect road hazards in a real road setting.

公开(公告)号：US20250118067A1

公开(公告)日：2025-04-10

申请号：US18887626

申请日：2024-09-17

Applicant: NEC Laboratories America, Inc.

Inventor： Sparsh Garg ,  Samuel Schulter ,  Yumin Suh

IPC: G06V10/98 ,  G06V10/26 ,  G06V10/762 ,  G06V10/82

Abstract: Systems and methods include generating a detection output for an image over multiple iterations by applying a dropout randomly to a different convolutional layer of a learning model for each iteration. The detection outputs are clustered, on labels, for each iteration. A total surface area for the clusters is computed over the iteration. A confidence is computed for the image using the total surface area for the clusters as an uncertainty score. A system is disabled if the confidence is below a threshold.

公开(公告)号：US20250118044A1

公开(公告)日：2025-04-10

申请号：US18891590

申请日：2024-09-20

Applicant: NEC Laboratories America, Inc.

Inventor： Jong-Chyi Su ,  Samuel Schulter ,  Sparsh Garg ,  Manmohan Chandraker ,  Mingfu Liang

IPC: G06V10/25 ,  G06V10/44 ,  G06V10/74 ,  G06V20/70

Abstract: Systems and methods for identifying novel objects in an image include detecting one or more objects in an image and generating one or more captions for the image. One or more predicted categories of the one or more objects detected in the image and the one or more captions are matched to identify, from the one or more predicted categories, a category of a novel object in the image. An image feature and a text description feature are generated using a description of the novel object. A relevant image is selected using a similarity score between the image feature and the text description feature. A model is updated using the relevant image and associated description of the novel object.