公开(公告)号：US12254681B2

公开(公告)日：2025-03-18

申请号：US17903393

申请日：2022-09-06

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Bingbing Zhuang ,  Samuel Schulter ,  Buyu Liu ,  Sparsh Garg ,  Ramin Moslemi ,  Inkyu Shin

IPC: G06K9/00 ,  G01S17/89 ,  G06V10/776 ,  G06V10/80

Abstract: Systems and methods are provided for multi-modal test-time adaptation. The method includes inputting a digital image into a pre-trained Camera Intra-modal Pseudo-label Generator, and inputting a point cloud set into a pre-trained Lidar Intra-modal Pseudo-label Generator. The method further includes applying a fast 2-dimension (2D) model, and a slow 2D model, to the inputted digital image to apply pseudo-labels, and applying a fast 3-dimension (3D) model, and a slow 3D model, to the inputted point cloud set to apply pseudo-labels. The method further includes fusing pseudo-label predictions from the fast models and the slow models through an Inter-modal Pseudo-label Refinement module to obtain robust pseudo labels, and measuring a prediction consistency for the pseudo-labels. The method further includes selecting confident pseudo-labels from the robust pseudo labels and measured prediction consistencies to form a final cross-modal pseudo-label set as a self-training signal, and updating batch parameters utilizing the self-training signal.

公开(公告)号：US20240354921A1

公开(公告)日：2024-10-24

申请号：US18616396

申请日：2024-03-26

Applicant: NEC Laboratories America, Inc.

Inventor： Sparsh Garg ,  Bingbing Zhuang ,  Samuel Schulter ,  Manmohan Chandraker

IPC: G06T7/00 ,  G06T7/10 ,  G06T7/50 ,  G06V20/56

CPC classification number: G06T7/0002 ,  G06T7/10 ,  G06T7/50 ,  G06V20/588 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30256

Abstract: Systems and methods for road defect level prediction. A depth map is obtained from an image dataset received from input peripherals by employing a vision transformer model. A plurality of semantic maps is obtained from the image dataset by employing a semantic segmentation model to give pixel-wise segmentation results of road scenes to detect road pixels. Regions of interest (ROI) are detected by utilizing the road pixels. Road defect levels are predicted by fitting the ROI and the depth map into a road surface model to generate road points classified into road defect levels. The predicted road defect levels are visualized on a road map.

公开(公告)号：US20250118063A1

公开(公告)日：2025-04-10

申请号：US18891625

申请日：2024-09-20

Applicant: NEC Laboratories America, Inc.

Inventor： Jong-Chyi Su ,  Samuel Schulter ,  Sparsh Garg ,  Manmohan Chandraker ,  Mingfu Liang

IPC: G06V10/82 ,  G06V10/75 ,  G06V20/70

Abstract: Systems and methods include detecting one or more objects in an image and generating one or more captions for the image. One or more predicted categories of the one or more objects detected in the image and the one or more captions are matched. From the one or more predicted categories, a category that is not successfully predicted in the image is identified. Data is curated to improve the category that is not successfully predicted in the image. A perception model is finetuned using data curated.

公开(公告)号：US12205356B2

公开(公告)日：2025-01-21

申请号：US18188766

申请日：2023-03-23

Applicant: NEC Laboratories America, Inc.

Inventor： Samuel Schulter ,  Sparsh Garg ,  Manmohan Chandraker

IPC: G06V10/776 ,  G06T7/00 ,  G06T7/11 ,  G06V10/74 ,  G06V10/774 ,  G06V20/70 ,  H04N17/00

Abstract: Methods and systems for detecting faults include capturing an image of a scene using a camera. The image is embedded using a segmentation model that includes an image branch having an image embedding layer that embeds images into a joint latent space and a text branch having a text embedding layer that embeds text into the joint latent space. Semantic information is generated for a region of the image corresponding to a predetermined static object using the embedded image. A fault of the camera is identified based on a discrepancy between the semantic information and semantic information of the predetermined static image. The fault of the camera is corrected.

公开(公告)号：US12045992B2

公开(公告)日：2024-07-23

申请号：US17520207

申请日：2021-11-05

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Masoud Faraki ,  Yumin Suh ,  Sparsh Garg ,  Manmohan Chandraker ,  Dongwan Kim

IPC: G06K9/00 ,  G06F18/214 ,  G06F18/2415 ,  G06F18/2431 ,  G06T7/11

CPC classification number: G06T7/11 ,  G06F18/2148 ,  G06F18/2415 ,  G06F18/2431 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods and systems for training a model include combining data from multiple datasets, the datasets having different respective label spaces. Relationships between labels in the different label spaces are identified. A unified neural network model is trained, using the combined data and the identified relationships to generate a unified model, with a class relational binary cross-entropy loss.

公开(公告)号：US20230281999A1

公开(公告)日：2023-09-07

申请号：US18188701

申请日：2023-03-23

Applicant: NEC Laboratories America, Inc.

Inventor： Samuel Schulter ,  Sparsh Garg

IPC: G06V20/54 ,  G06V20/70 ,  G06V10/774 ,  G06V10/26 ,  G06V10/74 ,  G06V10/86 ,  G08G1/16 ,  G08G1/09

CPC classification number: G06V20/54 ,  G06V20/70 ,  G06V10/774 ,  G06V10/26 ,  G06V10/761 ,  G06V10/86 ,  G08G1/16 ,  G08G1/09 ,  G06V10/82

Abstract: Methods and systems identifying road hazards include capturing an image of a road scene using a camera. The image is embedded using a segmentation model that includes an image branch having an image embedding layer that embeds images into a joint latent space and a text branch having a text embedding layer that embeds text into the joint latent space. A mask is generated for an object within the image using the segmentation model. A probability is determined that the object matches a road hazard using the segmentation mode. A signal is generated responsive to the probability to ameliorate a danger posed by the road hazard.

公开(公告)号：US20230073055A1

公开(公告)日：2023-03-09

申请号：US17903383

申请日：2022-09-06

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Sparsh Garg ,  Manmohan Chandraker ,  Samuel Shulter ,  Vijay Kumar Baikampady Gopalkrishna

IPC: G06T7/11

Abstract: A computer-implemented method for rut detection is provided. The method includes detecting, by a rut detection system, areas in a road-scene image that include ruts with pixel-wise probability values, wherein a higher value indicates a better chance of being a rut. The method further includes performing at least one of rut repair and vehicle rut avoidance responsive to the pixel-wise probability values. The detecting step includes performing neural network-based, pixel-wise semantic segmentation with context information on the road-scene image to distinguish rut pixels from non-rut pixels on a road depicted in the road-scene image.