公开(公告)号：US20230059370A1

公开(公告)日：2023-02-23

申请号：US17886747

申请日：2022-08-12

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Xinwei Shi ,  Anne Hobbs Dorsey ,  Rui Yan ,  Chi Yeung Jonathan Ng

IPC: G06V20/58 ,  G06V10/82 ,  G06T7/70 ,  G06V10/80 ,  B60W60/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predicting gaze and awareness using a neural network model. One of the methods includes obtaining sensor data (i) that is captured by one or more sensors of an autonomous vehicle and (ii) that characterizes an agent that is in a vicinity of the autonomous vehicle in an environment at a current time point. The sensor data is processed using a gaze prediction neural network to generate a gaze prediction that predicts a gaze of the agent at the current time point. The gaze prediction neural network includes an embedding subnetwork that is configured to process the sensor data to generate an embedding characterizing the agent, and a gaze subnetwork that is configured to process the embedding to generate the gaze prediction.