公开(公告)号：US12154212B2

公开(公告)日：2024-11-26

申请号：US17098943

申请日：2020-11-16

Applicant: Waymo LLC

Inventor： Zhenpei Yang ,  Yuning Chai ,  Yin Zhou ,  Pei Sun ,  Henrik Kretzschmar ,  Sean Rafferty ,  Dumitru Erhan ,  Dragomir Anguelov

IPC: G06T17/05 ,  B60W30/095 ,  B60W50/06 ,  B60W60/00 ,  G06N3/045 ,  G06N3/08 ,  G06T7/70 ,  G06T9/00 ,  G06T15/00 ,  G06T15/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generated simulated sensor data. One of the methods includes obtaining a surfel map generated from sensor observations of a real-world environment and generating, for each surfel in the surfel map, a respective grid having a plurality of grid cells, wherein each grid has an orientation matching an orientation of a corresponding surfel, and wherein each grid cell within each grid is assigned a respective color value. For a simulated location within a simulated representation of the real-world environment, a textured surfel rendering is generated, including combining color information from grid cells visible from the simulated location within the simulated representation of the real-world environment.

公开(公告)号：US20240232647A9

公开(公告)日：2024-07-11

申请号：US18492646

申请日：2023-10-23

Applicant: Waymo LLC

Inventor： Zhaoqi Leng ,  Guowang Li ,  Chenxi Liu ,  Pei Sun ,  Tong He ,  Dragomir Anguelov ,  Mingxing Tan

IPC: G06N3/0985

CPC classification number: G06N3/0985

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a machine learning model on training data. In one aspect, one of the methods include: obtaining a training data set comprising a plurality of training inputs; obtaining data defining an original search space of a plurality of candidate data augmentation policies; generating, from the original search space, a compact search space that has one or more global hyperparameters; and training the machine learning model on the training data using one or more final data augmentation policies generated from the compact search space.

公开(公告)号：US20240135195A1

公开(公告)日：2024-04-25

申请号：US18492646

申请日：2023-10-22

Applicant: Waymo LLC

Inventor： Zhaoqi Leng ,  Guowang Li ,  Chenxi Liu ,  Pei Sun ,  Tong He ,  Dragomir Anguelov ,  Mingxing Tan

IPC: G06N3/0985

CPC classification number: G06N3/0985

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a machine learning model on training data. In one aspect, one of the methods include: obtaining a training data set comprising a plurality of training inputs; obtaining data defining an original search space of a plurality of candidate data augmentation policies; generating, from the original search space, a compact search space that has one or more global hyperparameters; and training the machine learning model on the training data using one or more final data augmentation policies generated from the compact search space.

公开(公告)号：US11967103B2

公开(公告)日：2024-04-23

申请号：US17505900

申请日：2021-10-20

Applicant: Waymo LLC

Inventor： Jingxiao Zheng ,  Xinwei Shi ,  Alexander Gorban ,  Junhua Mao ,  Andre Liang Cornman ,  Yang Song ,  Ting Liu ,  Ruizhongtai Qi ,  Yin Zhou ,  Congcong Li ,  Dragomir Anguelov

IPC: G06T7/73 ,  G06F18/214 ,  G06F18/25 ,  G06V20/58

CPC classification number: G06T7/73 ,  G06F18/214 ,  G06F18/251 ,  G06V20/58 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30196 ,  G06T2207/30261

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for estimating a 3-D pose of an object of interest from image and point cloud data. In one aspect, a method includes obtaining an image of an environment; obtaining a point cloud of a three-dimensional region of the environment; generating a fused representation of the image and the point cloud; and processing the fused representation using a pose estimation neural network and in accordance with current values of a plurality of pose estimation network parameters to generate a pose estimation network output that specifies, for each of multiple keypoints, a respective estimated position in the three-dimensional region of the environment.

公开(公告)号：US20240096076A1

公开(公告)日：2024-03-21

申请号：US17945325

申请日：2022-09-15

Applicant: Waymo LLC

Inventor： Yin Zhou ,  Ruizhongtai Qi ,  Dragomir Anguelov ,  Minghua Liu ,  Boqing Gong

IPC: G06V10/82 ,  G06V10/80

CPC classification number: G06V10/82 ,  G06V10/80

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a semantic segmentation neural network for point clouds. One of the methods includes: obtaining a plurality of training points divided into a respective plurality of components; obtaining, for each of the respective plurality of components, data identifying a ground truth category for one or more labeled point; processing each training points using a semantic segmentation neural network to generate a semantic segmentation that includes a respective score for each of the plurality of categories; determining a gradient of a loss function that penalizes the semantic segmentation neural network for generating, for points in the component, non-zero scores for categories that are not the ground truth category for any labeled point in the component; and updating, using the gradient, the parameters of the semantic segmentation neural network.

公开(公告)号：US20230082365A1

公开(公告)日：2023-03-16

申请号：US17947046

申请日：2022-09-16

Applicant: Waymo LLC

Inventor： Kyriacos Christoforos Shiarlis ,  Dragomir Anguelov ,  Brandyn Allen White ,  Shimon Azariah Whiteson ,  Maximilian Igl ,  Daewoo Kim ,  Alex Richard Kuefler ,  Paul Marie Vincent Mougin ,  Punit Nilesh Shah ,  Mark Palatucci

IPC: B60W60/00 ,  G05B13/02 ,  B60W50/06

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating simulated trajectories using parallel beam search.

公开(公告)号：US20220391616A1

公开(公告)日：2022-12-08

申请号：US17341255

申请日：2021-06-07

Applicant: Waymo LLC

Inventor： Lo Po Tsui ,  Allen Minich ,  Megan Van Welie ,  Neil Barooah ,  Dragomir Anguelov

IPC: G06K9/00 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium that validates labels associated with sensor measurements of a scene in an environment. One of the methods includes receiving data representing a sensor measurement of a scene in an environment generated by one or more sensors. The sensor measurement can be associated with one or more labels, and each label can identify a portion of the sensor measurement that has been classified as measuring an object in the environment. For each of the labels, a determination can be made as to whether the label satisfies each of the validation criteria. Each validation criterion can measure whether one or more characteristics of the label are consistent with one or more characteristics of real-world objects in the environment. In response to determining that a particular label of the one or more labels does not satisfy one or more of the validation criteria, a notification can be generated indicating that the particular label is not a valid label for any real-world object in the scene of the environment.

公开(公告)号：US11417110B2

公开(公告)日：2022-08-16

申请号：US17015809

申请日：2020-09-09

Applicant: Waymo LLC

Inventor： Dragomir Anguelov ,  Colin Andrew Braley ,  Christoph Sprunk

IPC: G06V20/56 ,  G06K9/62

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for the generation and use of a surfel map with semantic labels. One of the methods includes receiving a surfel map that includes a plurality of surfels, wherein each surfel has associated data that includes one or more semantic labels; obtaining sensor data for one or more locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining one or more surfels corresponding to the one or more locations of the obtained sensor data; identifying one or more semantic labels for the one or more surfels corresponding to the one or more locations of the obtained sensor data; and performing, for each surfel corresponding to the one or more locations of the obtained sensor data, a label-specific detection process for the surfel.

公开(公告)号：US20220084228A1

公开(公告)日：2022-03-17

申请号：US17472418

申请日：2021-09-10

Applicant: Waymo LLC

Inventor： Alexander Gorban ,  Yin Zhou, JR. ,  Dragomir Anguelov ,  Alessandro Giulianelli

IPC: G06T7/521 ,  G06T7/73 ,  G01S17/89 ,  G01S7/48 ,  G05D1/00 ,  G01S17/931

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining estimated ground truth object keypoint labels for sensor readings of objects. In one aspect, a method comprises obtaining a plurality of sets of label data for a sensor reading of an object; obtaining respective quality control data corresponding to each of the plurality of sets of label data, the respective quality control data comprising: data indicating whether the labeled location of the first object keypoint in the corresponding set of label data is accurate; and determining an estimated ground truth location for the first object keypoint in the sensor data keypoint from (i) the labeled locations that were indicated as accurate by the corresponding quality control data and (ii) not from the labeled locations that were indicated as not accurate by the corresponding quality control data.

公开(公告)号：US20220058818A1

公开(公告)日：2022-02-24

申请号：US17407795

申请日：2021-08-20

Applicant: Waymo LLC

Inventor： Ruizhongtai Qi ,  Yin Zhou ,  Dragomir Anguelov ,  Pei Sun

IPC: G06T7/521 ,  G06T7/20

Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for performing three-dimensional auto-labeling on sensor data. The system obtains a sensor data segment that includes a temporal sequence of three-dimensional point clouds generated from sensor readings of an environment by one or more sensors. The system identifies, from the sensor data segment, (i) a plurality of object tracks that each corresponds to a different object in the environment and (ii) for each object track, respective initial three-dimensional regions in each of one or more of the point clouds in which the corresponding object appears. The system generates, for each object track, extracted object track data that includes at least the points in the respective initial three-dimensional regions for the object track. The system further generates, for each object track and from the extracted object track data for the object track, an auto labeling output that defines respective refined three-dimensional regions in each of the one or more point clouds.