-
公开(公告)号:US20230334842A1
公开(公告)日:2023-10-19
申请号:US18136252
申请日:2023-04-18
Applicant: Waymo LLC
Inventor: Alex Zihao Zhu , Vincent Michael Casser , Henrik Kretzschmar , Reza Mahjourian , Soeren Pirk
IPC: G06V10/82 , G06V10/774
CPC classification number: G06V10/82 , G06V10/774
Abstract: Methods, systems, and apparatus for processing inputs that include video frames using neural networks. In one aspect, a system comprises one or more computers configured to obtain a set of one or more training images and, for each training image, ground truth instance data that identifies, for each of one or more object instances, a corresponding region of the training image that depicts the object instance. For each training image in the set, the one or more computers process the training image using an instance segmentation neural network to generate an embedding output comprising a respective embedding for each of a plurality of output pixels. The one or more computers then train the instance segmentation neural network to minimize a loss function.
-
公开(公告)号:US20220358314A1
公开(公告)日:2022-11-10
申请号:US17314925
申请日:2021-05-07
Applicant: Waymo LLC
Inventor: Yulai Shen , Henrik Kretzschmar , Jeffrey Sham , Jeffrey Carlson , Lo Po Tsui , Dragomir Anguelov
Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for generating and editing object track labels for objects detected in video data. One of the methods includes obtaining a video segment comprising multiple image frames associated with multiple time points; obtaining object track data specifying a set of object tracks; providing, for presentation to a user, a user interface for modifying the object track data, the user interface displaying object timeline representations of the object tracks; receiving one or more user inputs that indicate one or more modifications to the object timeline representations; updating the object timeline representations displayed in the timeline display area; and updating the object track data according to the updated object timeline representations.
-
公开(公告)号:US20210150799A1
公开(公告)日:2021-05-20
申请号:US17098943
申请日:2020-11-16
Applicant: Waymo LLC
Inventor: Zhenpei Yang , Yuning Chai , Yin Zhou , Pei Sun , Henrik Kretzschmar , Sean Rafferty , Dumitru Erhan , Dragomir Anguelov
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generated simulated sensor data. One of the methods includes obtaining a surfel map generated from sensor observations of a real-world environment and generating, for each surfel in the surfel map, a respective grid having a plurality of grid cells, wherein each grid has an orientation matching an orientation of a corresponding surfel, and wherein each grid cell within each grid is assigned a respective color value. For a simulated location within a simulated representation of the real-world environment, a textured surfel rendering is generated, including combining color information from grid cells visible from the simulated location within the simulated representation of the real-world environment.
-
14.
公开(公告)号:US20250014357A1
公开(公告)日:2025-01-09
申请号:US18892711
申请日:2024-09-23
Applicant: Waymo LLC
Inventor: Ruichi Yu , Kang Li , Tao Han , Robert Cosgriff , Henrik Kretzschmar
IPC: G06V20/58 , G01S13/931 , G01S17/931 , G06F18/22
Abstract: Aspects of the disclosure relate to controlling a vehicle. For instance, using a camera, a first camera image including a first object may be captured. A first bounding box for the first object and a distance to the first object may be identified. A second camera image including a second object may be captured. A second bounding box for the second image and a distance to the second object may be identified. Whether the first object is the second object may be determined using a plurality of models to compare visual similarity of the two bounding boxes, to compare a three-dimensional location based on the distance to the first object and a three-dimensional location based on the distance to the second object, and to compare results from the first and second models. The vehicle may be controlled in an autonomous driving mode based on a result of the third model.
-
公开(公告)号:US12154212B2
公开(公告)日:2024-11-26
申请号:US17098943
申请日:2020-11-16
Applicant: Waymo LLC
Inventor: Zhenpei Yang , Yuning Chai , Yin Zhou , Pei Sun , Henrik Kretzschmar , Sean Rafferty , Dumitru Erhan , Dragomir Anguelov
IPC: G06T17/05 , B60W30/095 , B60W50/06 , B60W60/00 , G06N3/045 , G06N3/08 , G06T7/70 , G06T9/00 , G06T15/00 , G06T15/04
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generated simulated sensor data. One of the methods includes obtaining a surfel map generated from sensor observations of a real-world environment and generating, for each surfel in the surfel map, a respective grid having a plurality of grid cells, wherein each grid has an orientation matching an orientation of a corresponding surfel, and wherein each grid cell within each grid is assigned a respective color value. For a simulated location within a simulated representation of the real-world environment, a textured surfel rendering is generated, including combining color information from grid cells visible from the simulated location within the simulated representation of the real-world environment.
-
公开(公告)号:US20230281824A1
公开(公告)日:2023-09-07
申请号:US18118705
申请日:2023-03-07
Applicant: Waymo LLC
Inventor: Jieru Mei , Hang Yan , Liang-Chieh Chen , Siyuan Qiao , Yukun Zhu , Alex Zihao Zhu , Xinchen Yan , Henrik Kretzschmar
CPC classification number: G06T7/11 , G06V20/64 , G06V10/82 , G06V20/58 , G01S17/89 , G06T2207/10028 , G06T2207/20081 , G06T2207/30252 , G06T2207/10016 , G06T2210/12
Abstract: Methods, systems, and apparatus for generating a panoptic segmentation label for a sensor data sample. In one aspect, a system comprises one or more computers configured to obtain a sensor data sample characterizing a scene in an environment. The one or more computers obtain a 3D bounding box annotation at each time point for a point cloud characterizing the scene at the time point. The one or more computers obtain, for each camera image and each time point, annotation data identifying object instances depicted in the camera image, and the one or more computers generate a panoptic segmentation label for the sensor data sample characterizing the scene in the environment.
-
公开(公告)号:US20210150349A1
公开(公告)日:2021-05-20
申请号:US17099634
申请日:2020-11-16
Applicant: Waymo LLC
Inventor: Wei-Chih Hung , Henrik Kretzschmar , Yuning Chai , Dragomir Anguelov
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for multi object tracking using memory attention.
-
公开(公告)号:US10902272B2
公开(公告)日:2021-01-26
申请号:US16879299
申请日:2020-05-20
Applicant: Waymo LLC
Inventor: Victoria Dean , Abhijit S. Ogale , Henrik Kretzschmar , David Harrison Silver , Carl Kershaw , Pankaj Chaudhari , Chen Wu , Congcong Li
Abstract: Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.
-
公开(公告)号:US20190392231A1
公开(公告)日:2019-12-26
申请号:US16018490
申请日:2018-06-26
Applicant: Waymo LLC
Inventor: Victoria Dean , Abhijit S. Ogale , Henrik Kretzschmar , David Harrison Silver , Carl Kershaw , Pankaj Chaudhari , Chen Wu , Congcong Li
Abstract: Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.
-
-
-
-
-
-
-
-