MULTIMODAL 3D OBJECT DETECTION USING TEMPORAL AND STRUCTURE CONSISTENCY IN VOXEL FEATURE SPACE

    公开(公告)号:US20250166216A1

    公开(公告)日:2025-05-22

    申请号:US18516590

    申请日:2023-11-21

    Abstract: An example device for detecting objects through processing of media data, such as image data and point cloud data, includes a processing system configured to form voxel representations of a real-world three-dimensional (3D) space using images and point clouds captured for the 3D space at consecutive time steps, extract image and/or point cloud features for voxels in voxel representations of the 3D space, determine correspondences between the voxels at consecutive time steps according to similarities between the extracted features, and determine positions of objects in the 3D space using the correspondences between the voxels. For example, the processing system may perform triangulation according to positions of a moving object to positions of the voxels at the time steps. In this manner, the processing system may generate an accurate bird's eye view (BEV) representation of the real-world 3D space.

    ASYNCHRONOUS MULTIMODAL FEATURE FUSION

    公开(公告)号:US20250157204A1

    公开(公告)日:2025-05-15

    申请号:US18509026

    申请日:2023-11-14

    Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving image data from an image sensor; receiving ranging data from a ranging sensor; embedding first spatial features of the image data with first temporal information associated with the image data; embedding second spatial features of the ranging data with second temporal information associated with the ranging data; determining first bird's-eye-view (BEV) features based on the first spatial features embedded with first temporal information; determining second BEV features based on the second spatial features embedded with second temporal information; and determining, based on the first and second BEV features, a feature set for processing by a transformer network. The feature set includes at least a portion of both the first and second BEV features. Other aspects and features are also claimed and described.

    FEATURE EXTRACTION AND ALIGNMENT FOR NAVIGATION APPLICATIONS

    公开(公告)号:US20250086977A1

    公开(公告)日:2025-03-13

    申请号:US18463709

    申请日:2023-09-08

    Abstract: This disclosure provides systems, methods, and devices for processing and aligning sensor data features for navigation. In a first aspect, a method is provided that includes determining, based on received sensor data, a first set of features for an area surrounding the vehicle. A second set of features for the area surrounding the vehicle may be determined based on an occupancy map for the area surrounding the vehicle. A third set of features may be determined that align the first set of features with the second set of features. The third set of features may align each of at least a subset of the second set of features with at least one corresponding feature from the first set of features. Other aspects and features are also claimed and described.

    BI-DIRECTIONAL INFORMATION FLOW AMONG UNITS OF AN AUTONOMOUS DRIVING SYSTEM

    公开(公告)号:US20250054285A1

    公开(公告)日:2025-02-13

    申请号:US18447785

    申请日:2023-08-10

    Abstract: A sensor data processing system includes various elements, including a perception unit that collects data representing positions of sensors on a vehicle and obtains environmental information around the vehicle via the sensors. The sensor data processing system also includes a feature fusion unit that combines the first environmental information from the sensors into first fused feature data representing first positions of objects around the vehicle, provides the first fused feature data to the object tracking unit, receives feedback for the first fused feature data from the object tracking unit, and combines second environmental information from the sensors using the feedback into second fused feature data representing second positions of objects around the vehicle. The sensor data processing system may then at least partially control operation of the vehicle using the second fused feature data.

    GRAPH NEURAL NETWORK (GNN) IMPLEMENTED MULTI-MODAL SPATIOTEMPORAL FUSION

    公开(公告)号:US20250086979A1

    公开(公告)日:2025-03-13

    申请号:US18463109

    申请日:2023-09-07

    Abstract: Systems that support graph neural network (GNN) implemented multi-modal spatiotemporal fusion are provided. Identifying and tracking an object in images captured by an imaging system is facilitated by generating a graph based on multimodal data received from a plurality of sensors. The graph encodes spatial components and spatial data associated with the images and encodes temporal data associated with the images. Pooled features are generated, through application of a first graph attention network (GAT), by pooling spatial features and temporal features. The spatial features are based on the spatial component and on the spatial relationship, and the temporal features are based on the temporal relationship. A three dimensional bounding box associated with the object is decoded by propagating the pooled features through a fully connected layer.

    NERF-BASED MULTI-SENSOR DATA FUSION FOR VEHICLE APPLICATIONS

    公开(公告)号:US20250029393A1

    公开(公告)日:2025-01-23

    申请号:US18356504

    申请日:2023-07-21

    Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving a plurality of image frames representative of a scene; receiving point cloud data representative of the scene; determining, using a NeRF model, a three-dimensional reconstruction of the scene based on the plurality of image frames; and outputting fused data that combines first BEV features of the three-dimensional reconstruction of the scene and second BEV features of the point cloud data. Other aspects and features are also claimed and described.

Patent Agency Ranking