公开(公告)号：US20210334651A1

公开(公告)日：2021-10-28

申请号：US17194115

申请日：2021-03-05

Applicant: Waymo LLC

Inventor： Zhaoqi Leng ,  Ekin Dogus Cubuk ,  Barret Zoph ,  Jiquan Ngiam ,  Congcong Li ,  Jonathon Shlens ,  Shuyang Cheng

IPC: G06N3/08 ,  G06F17/18 ,  G06K9/62 ,  G01S17/894

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model to perform a machine learning task by processing input data to the model. For example, the input data can include image, video, or point cloud data, and the task can be a perception task such as classification or detection task. In one aspect, the method includes receiving training data including a plurality of training inputs; receiving a plurality of data augmentation policy parameters that define different transformation operations for transforming training inputs before the training inputs are used to train the machine learning model; maintaining a plurality of candidate machine learning models; for each of the plurality of candidate machine learning models: repeatedly determining an augmented batch of training data; training the candidate machine learning model using the augmented batch of the training data; and updating the maintained data.

公开(公告)号：US11093819B1

公开(公告)日：2021-08-17

申请号：US15381389

申请日：2016-12-16

Applicant: Waymo LLC

Inventor： Congcong Li ,  Ury Zhilinsky ,  Yun Jiang ,  Zhaoyin Jia

IPC: G06N3/04 ,  G06N3/08

Abstract: Disclosed herein are neural networks for generating target classifications for an object from a set of input sequences. Each input sequence includes a respective input at each of multiple time steps, and each input sequence corresponds to a different sensing subsystem of multiple sensing subsystems. For each time step in the multiple time steps and for each input sequence in the set of input sequences, a respective feature representation is generated for the input sequence by processing the respective input from the input sequence at the time step using a respective encoder recurrent neural network (RNN) subsystem for the sensing subsystem that corresponds to the input sequence. For each time step in at least a subset of the multiple time steps, the respective feature representations are processed using a classification neural network subsystem to select a respective target classification for the object at the time step.

公开(公告)号：US20210192238A1

公开(公告)日：2021-06-24

申请号：US17123185

申请日：2020-12-16

Applicant: WAYMO LLC

Inventor： Victoria Dean ,  Abhijit S. Ogale ,  Henrik Kretzschmar ,  David Harrison Silver ,  Carl Kershaw ,  Pankaj Chaudhari ,  Chen Wu ,  Congcong Li

IPC: G06K9/00 ,  G05D1/00 ,  G06N3/08 ,  G06T11/20 ,  G06K9/34 ,  G06F40/30

Abstract: Aspects of the disclosure relate to training and using a phrase recognition model to identify phrases in images. As an example, a selected phrase list may include a plurality of phrases is received. Each phrase of the plurality of phrases includes text. An initial plurality of images may be received. A training image set may be selected from the initial plurality of images by identifying the phrase-containing images that include one or more phrases from the selected phrase list. Each given phrase-containing image of the training image set may be labeled with information identifying the one or more phrases from the selected phrase list included in the given phrase-containing images. The model may be trained based on the training image set such that the model is configured to, in response to receiving an input image, output data indicating whether a phrase of the plurality of phrases is included in the input image.

公开(公告)号：US20210191395A1

公开(公告)日：2021-06-24

申请号：US16723787

申请日：2019-12-20

Applicant: Waymo LLC

Inventor： Jiyang Gao ,  Junhua Mao ,  Yi Shen ,  Congcong Li ,  Chen Sun

IPC: G05D1/00 ,  G06K9/00 ,  G06K9/62 ,  G05D1/02 ,  B60W60/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating vehicle intent predictions using a neural network. One of the methods includes obtaining an input characterizing one or more vehicles in an environment; generating, from the input, features of each of the vehicles; and for each of the vehicles: processing the features of the vehicle using each of a plurality of intent-specific neural networks, wherein each of the intent-specific neural networks corresponds to a respective intent from a set of intents, and wherein each intent-specific neural network is configured to process the features of the vehicle to generate an output for the corresponding intent.

公开(公告)号：US10366502B1

公开(公告)日：2019-07-30

申请号：US15374884

申请日：2016-12-09

Applicant: Waymo LLC

Inventor： Congcong Li

IPC: G06K9/00 ,  G05D1/00 ,  G06T7/50 ,  G06N3/04 ,  G06N3/08 ,  G06T7/73 ,  B60R1/00 ,  G05D1/02

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating vehicle heading predictions from point cloud data using a neural network. One of the methods includes receiving a plurality of different projections of point cloud data, wherein the point cloud data represents different sensor measurements of electromagnetic radiation reflected off a vehicle. Each of the plurality of projections of point cloud data is provided as input to a neural network subsystem trained to receive projections of point cloud data for a vehicle and to generate one or more vehicle heading classifications as an output. At the output of the neural network subsystem, one or more vehicle heading predictions is received.

公开(公告)号：US12067471B2

公开(公告)日：2024-08-20

申请号：US17104921

申请日：2020-11-25

Applicant: Waymo LLC

Inventor： Jiyang Gao ,  Zijian Guo ,  Congcong Li ,  Xiaowei Li

IPC: G06F16/24 ,  B60W60/00 ,  G06F30/27 ,  G06N3/02 ,  G06N3/08 ,  G06V20/56 ,  G08G1/01

CPC classification number: G06N3/02 ,  B60W60/0027 ,  G06F30/27 ,  G06N3/08 ,  G06V20/56 ,  G08G1/0104 ,  B60W2554/4045 ,  B60W2554/4046

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for searching an autonomous vehicle sensor data repository. One of the methods includes maintaining a collection of sensor samples and one or more embeddings of each sensor sample. Each sensor sample is generated from sensor data at multiple time steps and characterizes an environment at each of the multiple time steps. Each embedding corresponds to a respective portion of the sensor sample and has been generated by an embedding neural network. A query specifying a query portion of a query sensor sample is received. A query embedding corresponding to the query portion of the query sensor sample is generated through the embedding neural network. A plurality of relevant sensor samples that have embeddings that are closest to the query embedding are identified as characterizing similar scenarios to the query portion of the query sensor sample.

公开(公告)号：US11967103B2

公开(公告)日：2024-04-23

申请号：US17505900

申请日：2021-10-20

Applicant: Waymo LLC

Inventor： Jingxiao Zheng ,  Xinwei Shi ,  Alexander Gorban ,  Junhua Mao ,  Andre Liang Cornman ,  Yang Song ,  Ting Liu ,  Ruizhongtai Qi ,  Yin Zhou ,  Congcong Li ,  Dragomir Anguelov

IPC: G06T7/73 ,  G06F18/214 ,  G06F18/25 ,  G06V20/58

CPC classification number: G06T7/73 ,  G06F18/214 ,  G06F18/251 ,  G06V20/58 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30196 ,  G06T2207/30261

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for estimating a 3-D pose of an object of interest from image and point cloud data. In one aspect, a method includes obtaining an image of an environment; obtaining a point cloud of a three-dimensional region of the environment; generating a fused representation of the image and the point cloud; and processing the fused representation using a pose estimation neural network and in accordance with current values of a plurality of pose estimation network parameters to generate a pose estimation network output that specifies, for each of multiple keypoints, a respective estimated position in the three-dimensional region of the environment.

公开(公告)号：US11861481B2

公开(公告)日：2024-01-02

申请号：US16726060

申请日：2019-12-23

Applicant: Waymo LLC

Inventor： Zijian Guo ,  Nichola Abdo ,  Junhua Mao ,  Congcong Li ,  Edward Stephen Walker, Jr.

IPC: G06F16/53 ,  G06N3/042 ,  G06F16/538 ,  G06F16/535 ,  G05D1/00 ,  G05D1/02 ,  G06N3/08 ,  G06N3/045

CPC classification number: G06N3/042 ,  G05D1/0088 ,  G05D1/0221 ,  G06F16/535 ,  G06F16/538 ,  G06N3/045 ,  G06N3/08 ,  G05D2201/0213

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for searching an autonomous vehicle sensor data repository. One of the methods includes maintaining a collection of sensor samples and, for each sensor sample, an embedding of the sensor sample; receiving a request specifying a query sensor sample, wherein the query sensor sample characterizes a query environment region; and identifying, from the collection of sensor samples, a plurality of relevant sensor samples that characterize similar environment regions to the query environment region, comprising: processing the query sensor sample through the embedding neural network to generate a query embedding; and identifying, from sensor samples in a subset of the sensor samples in the collection, a plurality of sensor samples that have embeddings that are closest to the query embedding.

公开(公告)号：US11842282B2

公开(公告)日：2023-12-12

申请号：US17836287

申请日：2022-06-09

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Congcong Li ,  Yang Song

IPC: G06V20/56 ,  G06N3/084 ,  G05D1/02 ,  G06N3/08 ,  G06F18/20 ,  G06F18/25 ,  G06F18/21 ,  G06F18/214 ,  G06F18/2431 ,  G06N3/044 ,  G06V30/19 ,  G06V30/24 ,  G06V10/764 ,  G06V10/80 ,  G06V10/82 ,  G06V20/58

CPC classification number: G06N3/084 ,  G05D1/0221 ,  G05D1/0231 ,  G06F18/217 ,  G06F18/2148 ,  G06F18/2431 ,  G06F18/25 ,  G06F18/285 ,  G06N3/044 ,  G06N3/08 ,  G06V10/764 ,  G06V10/809 ,  G06V10/82 ,  G06V20/56 ,  G06V20/58 ,  G06V30/19173 ,  G06V30/2504

Abstract: Aspects of the subject matter disclosed herein include methods, systems, and other techniques for training, in a first phase, an object classifier neural network with a first set of training data, the first set of training data including a first plurality of training examples, each training example in the first set of training data being labeled with a coarse-object classification; and training, in a second phase after completion of the first phase, the object classifier neural network with a second set of training data, the second set of training data including a second plurality of training examples, each training example in the second set of training data being labeled with a fine-object classification.

公开(公告)号：US11783568B2

公开(公告)日：2023-10-10

申请号：US17224763

申请日：2021-04-07

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Qian Yu ,  Congcong Li

IPC: G06N3/08 ,  G06V20/58 ,  G06V10/764 ,  G05D1/00 ,  G05D1/02 ,  G06F18/2413 ,  G06V10/82

CPC classification number: G06V10/764 ,  G05D1/0088 ,  G05D1/0221 ,  G05D1/0231 ,  G06F18/2413 ,  G06N3/08 ,  G06V10/82 ,  G06V20/58 ,  G05D2201/0213

Abstract: Some aspects of the subject matter disclosed herein include a system implemented on one or more data processing apparatuses. The system can include an interface configured to obtain, from one or more sensor subsystems, sensor data describing an environment of a vehicle, and to generate, using the sensor data, (i) one or more first neural network inputs representing sensor measurements for a particular object in the environment and (ii) a second neural network input representing sensor measurements for at least a portion of the environment that encompasses the particular object and additional portions of the environment that are not represented by the one or more first neural network inputs; and a convolutional neural network configured to process the second neural network input to generate an output, the output including a plurality of feature vectors that each correspond to a different one a plurality of regions of the environment.