-
公开(公告)号:US20230150550A1
公开(公告)日:2023-05-18
申请号:US17988701
申请日:2022-11-16
Applicant: Waymo LLC
Inventor: Xinwei Shi , Tian Lan , Jonathan Chandler Stroud , Zhishuai Zhang , Junhua Mao , Jeonhyung Kang , Khaled Refaat , Jiachen Li
CPC classification number: B60W60/00274 , B60W60/0015 , B60W50/0097 , B60W40/04 , G06N3/049 , G06N3/08 , B60W2554/4029 , B60W2554/4045 , B60W2554/4046 , B60W2554/408 , B60W2556/10
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent behavior prediction using keypoint data. One of the methods includes obtaining data characterizing a scene in an environment, the data comprising: (i) context data comprising data characterizing historical trajectories of a plurality of agents up to the current time point; and (ii) keypoint data for a target agent; processing the context data using a context data encoder neural network to generate a context embedding for the target agent; processing the keypoint data using a keypoint encoder neural network to generate a keypoint embedding for the target agent; generating a combined embedding for the target agent from the context embedding and the keypoint embedding; and processing the combined embedding using a decoder neural network to generate a behavior prediction output for the target agent that characterizes predicted behavior of the target agent after the current time point.
-
公开(公告)号:US11610423B2
公开(公告)日:2023-03-21
申请号:US17099642
申请日:2020-11-16
Applicant: Waymo LLC
Inventor: Junhua Mao , Jiyang Gao , Yukai Liu , Congcong Li , Zhishuai Zhang , Dragomir Anguelov
IPC: G06V40/10 , G06K9/62 , B60W30/095 , G06V10/25
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using spatio-temporal-interactive networks.
-
公开(公告)号:US20220156965A1
公开(公告)日:2022-05-19
申请号:US17505900
申请日:2021-10-20
Applicant: Waymo LLC
Inventor: Jingxiao Zheng , Xinwei Shi , Alexander Gorban , Junhua Mao , Andre Liang Cornman , Yang Song , Ting Liu , Ruizhongtai Qi , Yin Zhou , Congcong Li , Dragomir Anguelov
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for estimating a 3-D pose of an object of interest from image and point cloud data. In one aspect, a method includes obtaining an image of an environment; obtaining a point cloud of a three-dimensional region of the environment; generating a fused representation of the image and the point cloud; and processing the fused representation using a pose estimation neural network and in accordance with current values of a plurality of pose estimation network parameters to generate a pose estimation network output that specifies, for each of multiple keypoints, a respective estimated position in the three-dimensional region of the environment.
-
公开(公告)号:US20210294346A1
公开(公告)日:2021-09-23
申请号:US17343187
申请日:2021-06-09
Applicant: Waymo LLC
Inventor: Junhua Mao , Congcong Li , Alper Ayvaci , Chen Sun , Kevin Murphy , Ruichi Yu
IPC: G05D1/02 , G05D1/00 , G06K9/00 , G06K9/62 , B60W30/095 , G01S17/931
Abstract: Aspects of the disclosure relate to training and using a model for identifying actions of objects. For instance, LIDAR sensor data frames including an object bounding box corresponding to an object as well as an action label for the bounding box may be received. Each sensor frame is associated with a timestamp and is sequenced with respect to other sensor frames. Each given sensor data frame may be projected into a camera image of the object based on the timestamp associated with the given sensor data frame in order to provide fused data. The model may be trained using the fused data such that the model is configured to, in response to receiving fused data, the model outputs an action label for each object bounding box of the fused data. This output may then be used to control a vehicle in an autonomous driving mode.
-
公开(公告)号:US11514310B2
公开(公告)日:2022-11-29
申请号:US16231297
申请日:2018-12-21
Applicant: Waymo LLC
Inventor: Junhua Mao , Lo Po Tsui , Congcong Li , Edward Stephen Walker, Jr.
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a classifier to detect open vehicle doors. One of the methods includes obtaining a plurality of initial training examples, each initial training example comprising (i) a sensor sample from a collection of sensor samples and (ii) data classifying the sensor sample as characterizing a vehicle that has an open door; generating a plurality of additional training examples, comprising, for each initial training example: identifying, from the collection of sensor samples, one or more additional sensor samples that were captured less than a threshold amount of time before the sensor sample in the initial training example was captured; and training the machine learning classifier on first training data that includes the initial training examples and the additional training examples to generate updated weights for the machine learning classifier.
-
公开(公告)号:US20220374650A1
公开(公告)日:2022-11-24
申请号:US17836287
申请日:2022-06-09
Applicant: Waymo LLC
Inventor: Junhua Mao , Congcong Li , Yang Song
Abstract: Aspects of the subject matter disclosed herein include methods, systems, and other techniques for training, in a first phase, an object classifier neural network with a first set of training data, the first set of training data including a first plurality of training examples, each training example in the first set of training data being labeled with a coarse-object classification; and training, in a second phase after completion of the first phase, the object classifier neural network with a second set of training data, the second set of training data including a second plurality of training examples, each training example in the second set of training data being labeled with a fine-object classification.
-
公开(公告)号:US11480963B2
公开(公告)日:2022-10-25
申请号:US16723787
申请日:2019-12-20
Applicant: Waymo LLC
Inventor: Jiyang Gao , Junhua Mao , Yi Shen , Congcong Li , Chen Sun
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating vehicle intent predictions using a neural network. One of the methods includes obtaining an input characterizing one or more vehicles in an environment; generating, from the input, features of each of the vehicles; and for each of the vehicles: processing the features of the vehicle using each of a plurality of intent-specific neural networks, wherein each of the intent-specific neural networks corresponds to a respective intent from a set of intents, and wherein each intent-specific neural network is configured to process the features of the vehicle to generate an output for the corresponding intent.
-
公开(公告)号:US20210150199A1
公开(公告)日:2021-05-20
申请号:US17099642
申请日:2020-11-16
Applicant: Waymo LLC
Inventor: Junhua Mao , Jiyang Gao , Yukai Liu , Congcong Li , Zhishuai Zhang , Dragomir Anguelov
IPC: G06K9/00 , G06K9/32 , G06K9/62 , B60W30/095
Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using spatio-temporal-interactive networks.
-
公开(公告)号:US20230059370A1
公开(公告)日:2023-02-23
申请号:US17886747
申请日:2022-08-12
Applicant: Waymo LLC
Inventor: Junhua Mao , Xinwei Shi , Anne Hobbs Dorsey , Rui Yan , Chi Yeung Jonathan Ng
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predicting gaze and awareness using a neural network model. One of the methods includes obtaining sensor data (i) that is captured by one or more sensors of an autonomous vehicle and (ii) that characterizes an agent that is in a vicinity of the autonomous vehicle in an environment at a current time point. The sensor data is processed using a gaze prediction neural network to generate a gaze prediction that predicts a gaze of the agent at the current time point. The gaze prediction neural network includes an embedding subnetwork that is configured to process the sensor data to generate an embedding characterizing the agent, and a gaze subnetwork that is configured to process the embedding to generate the gaze prediction.
-
公开(公告)号:US20210191395A1
公开(公告)日:2021-06-24
申请号:US16723787
申请日:2019-12-20
Applicant: Waymo LLC
Inventor: Jiyang Gao , Junhua Mao , Yi Shen , Congcong Li , Chen Sun
Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating vehicle intent predictions using a neural network. One of the methods includes obtaining an input characterizing one or more vehicles in an environment; generating, from the input, features of each of the vehicles; and for each of the vehicles: processing the features of the vehicle using each of a plurality of intent-specific neural networks, wherein each of the intent-specific neural networks corresponds to a respective intent from a set of intents, and wherein each intent-specific neural network is configured to process the features of the vehicle to generate an output for the corresponding intent.
-
-
-
-
-
-
-
-
-