公开(公告)号：US20230081913A1

公开(公告)日：2023-03-16

申请号：US17903393

申请日：2022-09-06

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Bingbing Zhuang ,  Samuel Schulter ,  Buyu Liu ,  Sparsh Garg ,  Ramin Moslemi ,  Inkyu Shin

IPC: G06V10/80 ,  G01S17/89 ,  G06V10/776

Abstract: Systems and methods are provided for multi-modal test-time adaptation. The method includes inputting a digital image into a pre-trained Camera Intra-modal Pseudo-label Generator, and inputting a point cloud set into a pre-trained Lidar Intra-modal Pseudo-label Generator. The method further includes applying a fast 2-dimension (2D) model, and a slow 2D model, to the inputted digital image to apply pseudo-labels, and applying a fast 3-dimension (3D) model, and a slow 3D model, to the inputted point cloud set to apply pseudo-labels. The method further includes fusing pseudo-label predictions from the fast models and the slow models through an Inter-modal Pseudo-label Refinement module to obtain robust pseudo labels, and measuring a prediction consistency for the pseudo-labels. The method further includes selecting confident pseudo-labels from the robust pseudo labels and measured prediction consistencies to form a final cross-modal pseudo-label set as a self-training signal, and updating batch parameters utilizing the self-training signal.

公开(公告)号：US12254681B2

公开(公告)日：2025-03-18

申请号：US17903393

申请日：2022-09-06

Applicant: NEC Laboratories America, Inc.

Inventor： Yi-Hsuan Tsai ,  Bingbing Zhuang ,  Samuel Schulter ,  Buyu Liu ,  Sparsh Garg ,  Ramin Moslemi ,  Inkyu Shin

IPC: G06K9/00 ,  G01S17/89 ,  G06V10/776 ,  G06V10/80

Abstract: Systems and methods are provided for multi-modal test-time adaptation. The method includes inputting a digital image into a pre-trained Camera Intra-modal Pseudo-label Generator, and inputting a point cloud set into a pre-trained Lidar Intra-modal Pseudo-label Generator. The method further includes applying a fast 2-dimension (2D) model, and a slow 2D model, to the inputted digital image to apply pseudo-labels, and applying a fast 3-dimension (3D) model, and a slow 3D model, to the inputted point cloud set to apply pseudo-labels. The method further includes fusing pseudo-label predictions from the fast models and the slow models through an Inter-modal Pseudo-label Refinement module to obtain robust pseudo labels, and measuring a prediction consistency for the pseudo-labels. The method further includes selecting confident pseudo-labels from the robust pseudo labels and measured prediction consistencies to form a final cross-modal pseudo-label set as a self-training signal, and updating batch parameters utilizing the self-training signal.