ASYNCHRONOUS MULTIMODAL FEATURE FUSION

    公开(公告)号:US20250157204A1

    公开(公告)日:2025-05-15

    申请号:US18509026

    申请日:2023-11-14

    Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method of image processing includes receiving image data from an image sensor; receiving ranging data from a ranging sensor; embedding first spatial features of the image data with first temporal information associated with the image data; embedding second spatial features of the ranging data with second temporal information associated with the ranging data; determining first bird's-eye-view (BEV) features based on the first spatial features embedded with first temporal information; determining second BEV features based on the second spatial features embedded with second temporal information; and determining, based on the first and second BEV features, a feature set for processing by a transformer network. The feature set includes at least a portion of both the first and second BEV features. Other aspects and features are also claimed and described.

    EARLY FUSION OF NEURAL RAY GRAPH NETWORKS FOR MULTI-VIEW CAMERA SETUPS

    公开(公告)号:US20250157178A1

    公开(公告)日:2025-05-15

    申请号:US18506507

    申请日:2023-11-10

    Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, an image processing method includes receiving image frames; determining an ordered set of neural rays based on the image frames; determining a graph network that represents each neural ray of the ordered set of neural rays as a sequence of points; and determining a feature set based on the graph network. Each neural ray of the ordered set of neural rays represents three-dimensional positions of pixels of an image frame. Each point on the graph network is associated with a node of a plurality of nodes of the graph network. The feature set includes features of each of the image frames. Other aspects and features are also claimed and described.

    CYLINDRICAL PARTITIONING FOR THREE-DIMENSIONAL (3D) PERCEPTION OPERATIONS

    公开(公告)号:US20250139882A1

    公开(公告)日:2025-05-01

    申请号:US18498995

    申请日:2023-10-31

    Abstract: In some aspects of the disclosure, an apparatus includes a processing system that includes one or more processors and one or more memories coupled to the one or more processors. The processing system is configured to receive sensor data associated with a scene and to generate a cylindrical representation associated with the scene. The processing system is further configured to modify the cylindrical representation based on detecting a feature of the cylindrical representation being included in a first region of the cylindrical representation. Modifying the cylindrical representation includes relocating the feature from the first region to a second region that is different than the first region. The processing system is further configured to perform, based on the modified cylindrical representation, one or more three-dimensional (3D) perception operations associated with the scene.

    TRAINING A NEURAL NETWORK TO GENERATE A DENSE DEPTH MAP FROM IMAGE AND LIDAR DATA

    公开(公告)号:US20250095173A1

    公开(公告)日:2025-03-20

    申请号:US18467035

    申请日:2023-09-14

    Abstract: An example device for training a neural network includes a memory configured to store a neural network model for the neural network; and a processing system comprising one or more processors implemented in circuitry, the processing system being configured to: extract image features from an image of an area, the image features representing objects in the area; extract point cloud features from a point cloud representation of the area, the point cloud features representing the objects in the area; add Gaussian noise to a ground truth depth map for the area to generate a noisy ground truth depth map, the ground truth depth map representing accurate positions of the objects in the area; and train the neural network using the image features, the point cloud features, and the noisy ground truth depth map to generate a depth map.

    KERNELIZED BIRD’S EYE VIEW SEGMENTATION FOR MULTI-SENSOR PERCEPTION

    公开(公告)号:US20250086978A1

    公开(公告)日:2025-03-13

    申请号:US18466460

    申请日:2023-09-13

    Abstract: An apparatus includes a memory for storing image data and position data, wherein the image data comprises a set of two-dimensional (2D) camera images, and wherein the position data comprises a set of three-dimensional (3D) point cloud frames. The apparatus also includes processing circuitry in communication with the memory, wherein the processing circuitry is configured to convert the set of 2D camera images into a first 3D representation of a 3D environment corresponding to the image data and the position data, wherein the set of 3D point cloud frames comprises a second 3D representation of the 3D environment. The processing circuitry is also configured to generate, based on the first 3D representation and the second 3D representation, a set of bird's eye view (BEV) feature kernels in a continuous space; and generate, based on the set of BEV feature kernels, an output.

    FEATURE EXTRACTION AND ALIGNMENT FOR NAVIGATION APPLICATIONS

    公开(公告)号:US20250086977A1

    公开(公告)日:2025-03-13

    申请号:US18463709

    申请日:2023-09-08

    Abstract: This disclosure provides systems, methods, and devices for processing and aligning sensor data features for navigation. In a first aspect, a method is provided that includes determining, based on received sensor data, a first set of features for an area surrounding the vehicle. A second set of features for the area surrounding the vehicle may be determined based on an occupancy map for the area surrounding the vehicle. A third set of features may be determined that align the first set of features with the second set of features. The third set of features may align each of at least a subset of the second set of features with at least one corresponding feature from the first set of features. Other aspects and features are also claimed and described.

    BI-DIRECTIONAL INFORMATION FLOW AMONG UNITS OF AN AUTONOMOUS DRIVING SYSTEM

    公开(公告)号:US20250054285A1

    公开(公告)日:2025-02-13

    申请号:US18447785

    申请日:2023-08-10

    Abstract: A sensor data processing system includes various elements, including a perception unit that collects data representing positions of sensors on a vehicle and obtains environmental information around the vehicle via the sensors. The sensor data processing system also includes a feature fusion unit that combines the first environmental information from the sensors into first fused feature data representing first positions of objects around the vehicle, provides the first fused feature data to the object tracking unit, receives feedback for the first fused feature data from the object tracking unit, and combines second environmental information from the sensors using the feedback into second fused feature data representing second positions of objects around the vehicle. The sensor data processing system may then at least partially control operation of the vehicle using the second fused feature data.

    MEMORY ORIENTED GAUSSIAN PROCESS BASED MULTI-OBJECT TRACKING

    公开(公告)号:US20240428547A1

    公开(公告)日:2024-12-26

    申请号:US18339408

    申请日:2023-06-22

    Abstract: An apparatus for multi-object tracking determines a current representation of a current object in a current image. The apparatus computes a joint Gaussian distribution between the current representation of the current object and a previous representation stored in one or more memory buffers, wherein the previous representation was determined from a previous image. The apparatus updates the one or more memory buffers based on the joint Gaussian distribution. For example, the apparatus determines whether to remove or replace the previous representation in the one or more memory buffers based on values of a covariance matrix of the joint Gaussian distribution.

    TERRAIN-AWARE OBJECT DETECTION FOR VEHICLE APPLICATIONS

    公开(公告)号:US20240378911A1

    公开(公告)日:2024-11-14

    申请号:US18314579

    申请日:2023-05-09

    Abstract: This disclosure provides systems, methods, and devices for vehicle driving assistance systems that support image processing. In a first aspect, a method is provided to train a machine learning model using image data and position data to identify contact points and ground surface normal vectors. Image data is received that depicts an object, and position data for the object is also received, such as point cloud position information for various points along the object's exterior surface. Two sets of labels may then be determined based on the position data, with one set identifying where the object contacts a ground surface and another identifying at least one normal vector for the ground surface. The machine learning model may then be trained based on both sets of labels to determine three-dimensional bounding boxes, normal maps, or combinations thereof. Other aspects and features are also claimed and described.

Patent Agency Ranking