公开(公告)号：US20230150550A1

公开(公告)日：2023-05-18

申请号：US17988701

申请日：2022-11-16

Applicant: Waymo LLC

Inventor： Xinwei Shi ,  Tian Lan ,  Jonathan Chandler Stroud ,  Zhishuai Zhang ,  Junhua Mao ,  Jeonhyung Kang ,  Khaled Refaat ,  Jiachen Li

IPC: B60W60/00 ,  B60W50/00 ,  B60W40/04 ,  G06N3/04 ,  G06N3/08

CPC classification number: B60W60/00274 ,  B60W60/0015 ,  B60W50/0097 ,  B60W40/04 ,  G06N3/049 ,  G06N3/08 ,  B60W2554/4029 ,  B60W2554/4045 ,  B60W2554/4046 ,  B60W2554/408 ,  B60W2556/10

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent behavior prediction using keypoint data. One of the methods includes obtaining data characterizing a scene in an environment, the data comprising: (i) context data comprising data characterizing historical trajectories of a plurality of agents up to the current time point; and (ii) keypoint data for a target agent; processing the context data using a context data encoder neural network to generate a context embedding for the target agent; processing the keypoint data using a keypoint encoder neural network to generate a keypoint embedding for the target agent; generating a combined embedding for the target agent from the context embedding and the keypoint embedding; and processing the combined embedding using a decoder neural network to generate a behavior prediction output for the target agent that characterizes predicted behavior of the target agent after the current time point.

公开(公告)号：US20220156965A1

公开(公告)日：2022-05-19

申请号：US17505900

申请日：2021-10-20

Applicant: Waymo LLC

Inventor： Jingxiao Zheng ,  Xinwei Shi ,  Alexander Gorban ,  Junhua Mao ,  Andre Liang Cornman ,  Yang Song ,  Ting Liu ,  Ruizhongtai Qi ,  Yin Zhou ,  Congcong Li ,  Dragomir Anguelov

IPC: G06T7/73 ,  G06K9/62 ,  G06K9/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for estimating a 3-D pose of an object of interest from image and point cloud data. In one aspect, a method includes obtaining an image of an environment; obtaining a point cloud of a three-dimensional region of the environment; generating a fused representation of the image and the point cloud; and processing the fused representation using a pose estimation neural network and in accordance with current values of a plurality of pose estimation network parameters to generate a pose estimation network output that specifies, for each of multiple keypoints, a respective estimated position in the three-dimensional region of the environment.

公开(公告)号：US20250037303A1

公开(公告)日：2025-01-30

申请号：US18614254

申请日：2024-03-22

Applicant: Waymo LLC

Inventor： Jingxiao Zheng ,  Xinwei Shi ,  Alexander Gorban ,  Junhua Mao ,  Andre Liang Cornman ,  Yang Song ,  Ting Liu ,  Ruizhongtai Qi ,  Yin Zhou ,  Congcong Li ,  Dragomir Anguelov

IPC: G06T7/73 ,  G06F18/214 ,  G06F18/25 ,  G06V20/58

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for estimating a 3-D pose of an object of interest from image and point cloud data. In one aspect, a method includes obtaining an image of an environment; obtaining a point cloud of a three-dimensional region of the environment; generating a fused representation of the image and the point cloud; and processing the fused representation using a pose estimation neural network and in accordance with current values of a plurality of pose estimation network parameters to generate a pose estimation network output that specifies, for each of multiple keypoints, a respective estimated position in the three-dimensional region of the environment.

公开(公告)号：US20230059370A1

公开(公告)日：2023-02-23

申请号：US17886747

申请日：2022-08-12

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Xinwei Shi ,  Anne Hobbs Dorsey ,  Rui Yan ,  Chi Yeung Jonathan Ng

IPC: G06V20/58 ,  G06V10/82 ,  G06T7/70 ,  G06V10/80 ,  B60W60/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predicting gaze and awareness using a neural network model. One of the methods includes obtaining sensor data (i) that is captured by one or more sensors of an autonomous vehicle and (ii) that characterizes an agent that is in a vicinity of the autonomous vehicle in an environment at a current time point. The sensor data is processed using a gaze prediction neural network to generate a gaze prediction that predicts a gaze of the agent at the current time point. The gaze prediction neural network includes an embedding subnetwork that is configured to process the sensor data to generate an embedding characterizing the agent, and a gaze subnetwork that is configured to process the embedding to generate the gaze prediction.

公开(公告)号：US11967103B2

公开(公告)日：2024-04-23

申请号：US17505900

申请日：2021-10-20

Applicant: Waymo LLC

Inventor： Jingxiao Zheng ,  Xinwei Shi ,  Alexander Gorban ,  Junhua Mao ,  Andre Liang Cornman ,  Yang Song ,  Ting Liu ,  Ruizhongtai Qi ,  Yin Zhou ,  Congcong Li ,  Dragomir Anguelov

IPC: G06T7/73 ,  G06F18/214 ,  G06F18/25 ,  G06V20/58

CPC classification number: G06T7/73 ,  G06F18/214 ,  G06F18/251 ,  G06V20/58 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30196 ,  G06T2207/30261

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for estimating a 3-D pose of an object of interest from image and point cloud data. In one aspect, a method includes obtaining an image of an environment; obtaining a point cloud of a three-dimensional region of the environment; generating a fused representation of the image and the point cloud; and processing the fused representation using a pose estimation neural network and in accordance with current values of a plurality of pose estimation network parameters to generate a pose estimation network output that specifies, for each of multiple keypoints, a respective estimated position in the three-dimensional region of the environment.

公开(公告)号：US20230062158A1

公开(公告)日：2023-03-02

申请号：US17902670

申请日：2022-09-02

Applicant: Waymo LLC

Inventor： Xinwei Shi ,  Junhua Mao ,  Khaled Refaat ,  Tian Lan ,  Jeonhyung Kang ,  Zhishuai Zhang ,  Jonathan Chandler Stroud

IPC: B60W60/00 ,  B60W40/04 ,  B60W50/00 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium that determine yield behavior for an autonomous vehicle, and can include identifying an agent that is in a vicinity of an autonomous vehicle navigating through a scene at a current time point. Scene features can be obtained and can include features of (i) the agent and (ii) the autonomous vehicle. An input that can include the scene features can be processed using a first machine learning model that is configured to generate (i) a crossing intent prediction that includes a crossing intent score that represents a likelihood that the agent intends to cross a roadway in a future time window after the current time, and (ii) a crossing action prediction that includes a crossing action score that represents a likelihood that the agent will cross the roadway in the future time window after the current time.