End-to-end object tracking using neural networks with attention

    公开(公告)号:US12175767B2

    公开(公告)日:2024-12-24

    申请号:US17715838

    申请日:2022-04-07

    Applicant: Waymo LLC

    Abstract: The described aspects and implementations enable efficient calibration of a sensing system of a vehicle. In one implementation, disclosed is a method and a system to perform the method, the system including the sensing system configured to obtain a plurality of images associated with a corresponding time of a plurality of times. The system further includes a data processing system operatively coupled to the sensing system and configured to generate a plurality of sets of feature tensors (FTs) associated with one or more objects of the environment depicted in a respective image. The data processing system is further to obtain a combined FT and process the combined FT using a neural network to identify one or more tracks characterizing motion of a respective object.

    ASSOCIATION OF CAMERA IMAGES AND RADAR DATA IN AUTONOMOUS VEHICLE APPLICATIONS

    公开(公告)号:US20230038842A1

    公开(公告)日:2023-02-09

    申请号:US17444338

    申请日:2021-08-03

    Applicant: Waymo LLC

    Abstract: The described aspects and implementations enable fast and accurate object identification in autonomous vehicle (AV) applications by combining radar data with camera images. In one implementation, disclosed is a method and a system to perform the method that includes obtaining a radar image of a first hypothetical object in an environment of the AV, obtaining a camera image of a second hypothetical object in the environment of the AV, and processing the radar image and the camera image using one or more machine-learning models MLMs to obtain a prediction measure representing a likelihood that the first hypothetical object and the second hypothetical object correspond to a same object in the environment of the AV.

    THREE-DIMENSIONAL LOCATION PREDICTION FROM IMAGES

    公开(公告)号:US20220180549A1

    公开(公告)日:2022-06-09

    申请号:US17545987

    申请日:2021-12-08

    Applicant: Waymo LLC

    Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting three-dimensional object locations from images. One of the methods includes obtaining a sequence of images that comprises, at each of a plurality of time steps, a respective image that was captured by a camera at the time step; generating, for each image in the sequence, respective pseudo-lidar features of a respective pseudo-lidar representation of a region in the image that has been determined to depict a first object; generating, for a particular image at a particular time step in the sequence, image patch features of the region in the particular image that has been determined to depict the first object; and generating, from the respective pseudo-lidar features and the image patch features, a prediction that characterizes a location of the first object in a three-dimensional coordinate system at the particular time step in the sequence.

    STATEFUL AND END-TO-END MULTI-OBJECT TRACKING

    公开(公告)号:US20240303827A1

    公开(公告)日:2024-09-12

    申请号:US18600449

    申请日:2024-03-08

    Applicant: Waymo LLC

    Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for tracking objects in an environment across time. In one aspect, a method comprises: receiving a set of current object detections, each characterizing features of a respective detected object; maintaining data, including track query feature representations, that identifies one or more object tracks (each associated with respective earlier object detections classified as characterizing the same object; and, for each object track: (i) selecting a subset of the current object detections as candidate object detections for the object track, (ii) generating a respective association score for each candidate object detection based on an input derived from the candidate object detections and the track query feature representation for the object track using a track-detection interaction neural network, and (iii) determining whether to associate any of the current object detections with the object track based on the respective association scores.

    END-TO-END OBJECT TRACKING USING NEURAL NETWORKS WITH ATTENTION

    公开(公告)号:US20250078927A1

    公开(公告)日:2025-03-06

    申请号:US18953053

    申请日:2024-11-19

    Applicant: Waymo LLC

    Abstract: The described aspects and implementations enable efficient calibration of a sensing system of a vehicle. In one implementation, disclosed is a method and a system to perform the method, the system including the sensing system configured to obtain a plurality of images associated with a corresponding time of a plurality of times. The system further includes a data processing system operatively coupled to the sensing system and configured to generate a plurality of sets of feature tensors (FTs) associated with one or more objects of the environment depicted in a respective image. The data processing system is further to obtain a combined FT and process the combined FT using a neural network to identify one or more tracks characterizing motion of a respective object.

Patent Agency Ranking