公开(公告)号：US20230038842A1

公开(公告)日：2023-02-09

申请号：US17444338

申请日：2021-08-03

Applicant: Waymo LLC

Inventor： Ruichi Yu ,  Shiwei Sheng ,  Kang Li ,  Xu Chen

IPC: G06K9/00 ,  G06K9/62 ,  G06N20/20 ,  G01S13/86 ,  G01S7/41 ,  G01S13/931

Abstract: The described aspects and implementations enable fast and accurate object identification in autonomous vehicle (AV) applications by combining radar data with camera images. In one implementation, disclosed is a method and a system to perform the method that includes obtaining a radar image of a first hypothetical object in an environment of the AV, obtaining a camera image of a second hypothetical object in the environment of the AV, and processing the radar image and the camera image using one or more machine-learning models MLMs to obtain a prediction measure representing a likelihood that the first hypothetical object and the second hypothetical object correspond to a same object in the environment of the AV.

公开(公告)号：US11544869B2

公开(公告)日：2023-01-03

申请号：US17342434

申请日：2021-06-08

Applicant: Waymo LLC

Inventor： Alper Ayvaci ,  Yu-Han Chen ,  Ruichi Yu ,  Chen Wu ,  Noha Waheed Ahmed Radwan ,  Jonathon Shlens

IPC: G06T7/73 ,  G06T7/50 ,  G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating object interaction predictions using a neural network. One of the methods includes obtaining a sensor input derived from data generated by one or more sensors that characterizes a scene. The sensor input is provided to an object interaction neural network. The object interaction neural network is configured to process the sensor input to generate a plurality of object interaction outputs. Each respective object interaction output includes main object information and interacting object information. The respective object interaction outputs corresponding to the plurality of regions in the sensor input are received as output of the object interaction neural network.

公开(公告)号：US20220366175A1

公开(公告)日：2022-11-17

申请号：US17319194

申请日：2021-05-13

Applicant: WAYMO LLC

Inventor： Ruichi Yu ,  Kang Li ,  Tao Han ,  Robert Cosgriff ,  Henrik Kretzschmar

IPC: G06K9/00 ,  G06K9/62 ,  G05D1/00 ,  G05D1/02 ,  G01S17/931 ,  G01S13/931

Abstract: Aspects of the disclosure relate to controlling a vehicle. For instance, using a camera, a first camera image including a first object may be captured. A first bounding box for the first object and a distance to the first object may be identified. A second camera image including a second object may be captured. A second bounding box for the second image and a distance to the second object may be identified. Whether the first object is the second object may be determined using a plurality of models to compare visual similarity of the two bounding boxes, to compare a three-dimensional location based on the distance to the first object and a three-dimensional location based on the distance to the second object, and to compare results from the first and second models. The vehicle may be controlled in an autonomous driving mode based on a result of the third model.

公开(公告)号：US20220180549A1

公开(公告)日：2022-06-09

申请号：US17545987

申请日：2021-12-08

Applicant: Waymo LLC

Inventor： Longlong Jing ,  Ruichi Yu ,  Jiyang Gao ,  Henrik Kretzschmar ,  Kang Li ,  Ruizhongtai Qi ,  Hang Zhao ,  Alper Ayvaci ,  Xu Chen ,  Dillon Cower ,  Congcong Li

IPC: G06T7/70 ,  G06V10/40 ,  G06V10/80 ,  G06T7/50 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting three-dimensional object locations from images. One of the methods includes obtaining a sequence of images that comprises, at each of a plurality of time steps, a respective image that was captured by a camera at the time step; generating, for each image in the sequence, respective pseudo-lidar features of a respective pseudo-lidar representation of a region in the image that has been determined to depict a first object; generating, for a particular image at a particular time step in the sequence, image patch features of the region in the particular image that has been determined to depict the first object; and generating, from the respective pseudo-lidar features and the image patch features, a prediction that characterizes a location of the first object in a three-dimensional coordinate system at the particular time step in the sequence.

公开(公告)号：US11315260B2

公开(公告)日：2022-04-26

申请号：US16726053

申请日：2019-12-23

Applicant: Waymo LLC

Inventor： Ruichi Yu ,  Sachithra Madhawa Hemachandra ,  Ian James Mahon ,  Congcong Li

IPC: G06T7/246 ,  G06N3/04 ,  G06N3/08 ,  G05D1/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for associating a new measurement of an object surrounding a vehicle with a maintained track. One of the methods includes receiving an object track for a particular object, receiving a new measurement characterizing a new object at a new time step, and determining whether the new object is the same as the particular object, comprising: generating a representation of the new object at the new and preceding time steps; generating a representation of the particular object at the new and preceding time steps; processing a first network input comprising the representations using a first neural network to generate an embedding of the first network input; and processing the embedding of the first network input using a second neural network to generate a predicted likelihood that the new object and the particular object are the same.

公开(公告)号：US11061406B2

公开(公告)日：2021-07-13

申请号：US16167007

申请日：2018-10-22

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Congcong Li ,  Alper Ayvaci ,  Chen Sun ,  Kevin Murphy ,  Ruichi Yu

IPC: G05D1/02 ,  B60W30/095 ,  G01S17/93 ,  G05D1/00 ,  G06K9/00 ,  G06K9/62 ,  G01S17/931

Abstract: Aspects of the disclosure relate to training and using a model for identifying actions of objects. For instance, LIDAR sensor data frames including an object bounding box corresponding to an object as well as an action label for the bounding box may be received. Each sensor frame is associated with a timestamp and is sequenced with respect to other sensor frames. Each given sensor data frame may be projected into a camera image of the object based on the timestamp associated with the given sensor data frame in order to provide fused data. The model may be trained using the fused data such that the model is configured to, in response to receiving fused data, the model outputs an action label for each object bounding box of the fused data. This output may then be used to control a vehicle in an autonomous driving mode.