公开(公告)号：US20240161478A1

公开(公告)日：2024-05-16

申请号：US18130200

申请日：2023-04-03

Applicant: University of Science and Technology Beijing

Inventor： Huimin MA ,  Haizhuang LIU ,  Yilin WANG ,  Rongquan WANG

IPC: G06V10/80 ,  G06T5/00 ,  G06T7/73 ,  G06V10/77 ,  G06V10/774 ,  G06V20/58 ,  G06V20/64

CPC classification number: G06V10/803 ,  G06T5/002 ,  G06T7/73 ,  G06V10/7715 ,  G06V10/774 ,  G06V10/806 ,  G06V20/58 ,  G06V20/64 ,  G06T2207/10024 ,  G06T2207/10028 ,  G06T2207/20021 ,  G06T2207/20081 ,  G06T2207/30196 ,  G06T2207/30252

Abstract: Disclosed are a multimodal weakly-supervised three-dimensional (3D) object detection method and system, and a device. The method includes: shooting multiple two-dimensional (2D) red, green and blue (RGB) images with a camera, acquiring ground points by a vehicle LiDAR sensor and generating a 3D frustum based on 2D box labels on each of the 2D RGB images; filtering ground points in the 3D frustum and selecting a region with most 3D points; generating a 3D pseudo-labeling bounding box of an object according to the region with the most 3D points; training a multimodal superpixel dual-branch network with the 3D pseudo-labeling bounding boxes as labels and the 2D RGB image and the 3D point cloud as inputs; and inputting a 2D RGB image of a current frame and a 3D point cloud of a current scenario to a trained multimodal superpixel dual-branch network to generate an overall 3D point cloud.