公开(公告)号：US20240025454A1

公开(公告)日：2024-01-25

申请号：US18228365

申请日：2023-07-31

申请人： Waymo LLC

发明人： Khaled Refaat

IPC分类号： B60W60/00 ,  G01S17/931 ,  G01S17/58 ,  G01S13/58 ,  G01S13/931

CPC分类号： B60W60/0027 ,  G01S17/931 ,  G01S17/58 ,  G01S13/58 ,  G01S13/931 ,  B60W2555/00 ,  B60W2420/52

摘要： An autonomous vehicle includes sensor subsystem(s) that output a sensor signal. A perception subsystem (i) detects an agent in a vicinity of the autonomous vehicle and (ii) generates a motion signal that describes at least one of a past motion or a present motion of the agent. An intention prediction subsystem processes the sensor signal to generate an intention signal that describes at least one intended action of the agent. A behavior prediction subsystem processes the motion signal and the intention signal to generate a behavior prediction signal that describes at least one predicted behavior of the agent. A planner subsystem processes the behavior prediction signal to plan a driving decision for the autonomous vehicle.

公开(公告)号：US11873011B2

公开(公告)日：2024-01-16

申请号：US17083988

申请日：2020-10-29

申请人： Waymo LLC

发明人： Khaled Refaat ,  Kai Ding

IPC分类号： B60W60/00 ,  B60W40/04 ,  G06N20/00

CPC分类号： B60W60/0027 ,  B60W40/04 ,  G06N20/00

摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating candidate future trajectories for agents. One of the methods includes obtaining scene data characterizing a scene in an environment at a current time point; for each of a plurality of lane segments, processing a model input comprising (i) features of the lane segment and (ii) features of the target agent using a machine learning model that is configured to process the model input to generate a respective score for the lane segment that represents a likelihood that the lane segment will be a first lane segment traversed by the target agent after the current time point; selecting, as a set of seed lane segments, a proper subset of the plurality of lane segments based on the respective scores; and generating a plurality of candidate future trajectories for the target agent.

公开(公告)号：US20230406360A1

公开(公告)日：2023-12-21

申请号：US18335915

申请日：2023-06-15

申请人： Waymo LLC

发明人： Rami Al-Rfou ,  Nigamaa Nayakanti ,  Kratarth Goel ,  Aurick Qikun Zhou ,  Benjamin Sapp ,  Khaled Refaat

IPC分类号： B60W60/00 ,  B60W40/06 ,  B60W40/04 ,  G06N3/0455

CPC分类号： B60W60/0027 ,  B60W40/06 ,  B60W2556/10 ,  G06N3/0455 ,  B60W40/04

摘要： Methods, systems, and apparatus for generating trajectory predictions for one or more target agents. In one aspect, a system comprises one or more computers configured to obtain scene context data characterizing a scene in an environment at a current time point, where the scene includes multiple agents that include a target agent and one or more context agents, and the scene context data includes respective context data for each of multiple different modalities of context data. The one or more computers then generate an encoded representation of the scene in the environment that includes one or more embeddings and process the encoded representation of the scene context data using a decoder neural network to generate a trajectory prediction output for the target agent that predicts a future trajectory of the target after the current time point.

公开(公告)号：US11673550B2

公开(公告)日：2023-06-13

申请号：US17320727

申请日：2021-05-14

申请人： Waymo LLC

发明人： Kai Ding ,  Khaled Refaat ,  Stephane Ross

IPC分类号： B60W30/095 ,  G05D1/02 ,  G05D1/00

CPC分类号： B60W30/0956 ,  G05D1/0088 ,  G05D1/0214 ,  G05D2201/0213

摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying high-priority agents in the vicinity of a vehicle. In one aspect, a method comprises processing an input that characterizes a trajectory of the vehicle in an environment using an importance scoring model to generate an output that defines a respective importance score for each of a plurality of agents in the environment in the vicinity of the vehicle. The importance score for an agent characterizes an estimated impact of the agent on planning decisions generated by a planning system of the vehicle which plans a future trajectory of the vehicle. The high-priority agents are identified as a proper subset of the plurality of agents with the highest importance scores.

公开(公告)号：US20230062158A1

公开(公告)日：2023-03-02

申请号：US17902670

申请日：2022-09-02

申请人： Waymo LLC

发明人： Xinwei Shi ,  Junhua Mao ,  Khaled Refaat ,  Tian Lan ,  Jeonhyung Kang ,  Zhishuai Zhang ,  Jonathan Chandler Stroud

IPC分类号： B60W60/00 ,  B60W40/04 ,  B60W50/00 ,  G06N3/08

摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium that determine yield behavior for an autonomous vehicle, and can include identifying an agent that is in a vicinity of an autonomous vehicle navigating through a scene at a current time point. Scene features can be obtained and can include features of (i) the agent and (ii) the autonomous vehicle. An input that can include the scene features can be processed using a first machine learning model that is configured to generate (i) a crossing intent prediction that includes a crossing intent score that represents a likelihood that the agent intends to cross a roadway in a future time window after the current time, and (ii) a crossing action prediction that includes a crossing action score that represents a likelihood that the agent will cross the roadway in the future time window after the current time.

公开(公告)号：US11340622B2

公开(公告)日：2022-05-24

申请号：US16557938

申请日：2019-08-30

申请人： Waymo LLC

发明人： Khaled Refaat ,  Kai Ding ,  Stephane Ross

IPC分类号： G05D1/02 ,  G01C21/34

摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining respective importance scores for a plurality of agents in a vicinity of an autonomous vehicle navigating through an environment. The respective importance scores characterize a relative impact of each agent on planned trajectories generated by a planning subsystem of the autonomous vehicle. In one aspect, a method comprises providing different states of an environment as input to the planning subsystem and obtaining as output from the planning subsystem corresponding planned trajectories. Importance scores for the one or more agents that are in one state but not in the other are determined based on a measure of difference between the planned trajectories.

公开(公告)号：US20220135078A1

公开(公告)日：2022-05-05

申请号：US17083988

申请日：2020-10-29

申请人： Waymo LLC

发明人： Khaled Refaat ,  Kai Ding

IPC分类号： B60W60/00 ,  G06N20/00 ,  B60W40/04

摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating candidate future trajectories for agents. One of the methods includes obtaining scene data characterizing a scene in an environment at a current time point; for each of a plurality of lane segments, processing a model input comprising (i) features of the lane segment and (ii) features of the target agent using a machine learning model that is configured to process the model input to generate a respective score for the lane segment that represents a likelihood that the lane segment will be a first lane segment traversed by the target agent after the current time point; selecting, as a set of seed lane segments, a proper subset of the plurality of lane segments based on the respective scores; and generating a plurality of candidate future trajectories for the target agent.

公开(公告)号：US20210232147A1

公开(公告)日：2021-07-29

申请号：US17229384

申请日：2021-04-13

申请人： Waymo LLC

发明人： Khaled Refaat ,  Stephane Ross

IPC分类号： G05D1/02 ,  G06N20/00 ,  G05D1/00 ,  G06N5/04

摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a representation of a trajectory of a target agent in an environment. In one aspect, the representation of the trajectory of the target agent in the environment is a concatenation of a plurality of channels, where each channel is represented as a two-dimensional array of data values. Each position in each channel corresponds to a respective spatial position in the environment, and corresponding positions in different channels correspond to the same spatial position in the environment. The channels include a time channel and a respective motion channel corresponding to each motion parameter in a predetermined set of motion parameters.

公开(公告)号：US20210064044A1

公开(公告)日：2021-03-04

申请号：US16557938

申请日：2019-08-30

申请人： Waymo LLC

发明人： Khaled Refaat ,  Kai Ding ,  Stephane Ross

IPC分类号： G05D1/02 ,  G01C21/34

摘要： Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining respective importance scores for a plurality of agents in a vicinity of an autonomous vehicle navigating through an environment. The respective importance scores characterize a relative impact of each agent on planned trajectories generated by a planning subsystem of the autonomous vehicle. In one aspect, a method comprises providing different states of an environment as input to the planning subsystem and obtaining as output from the planning subsystem corresponding planned trajectories. Importance scores for the one or more agents that are in one state but not in the other are determined based on a measure of difference between the planned trajectories.

公开(公告)号：US20230150550A1

公开(公告)日：2023-05-18

申请号：US17988701

申请日：2022-11-16

申请人： Waymo LLC

发明人： Xinwei Shi ,  Tian Lan ,  Jonathan Chandler Stroud ,  Zhishuai Zhang ,  Junhua Mao ,  Jeonhyung Kang ,  Khaled Refaat ,  Jiachen Li

IPC分类号： B60W60/00 ,  B60W50/00 ,  B60W40/04 ,  G06N3/04 ,  G06N3/08

CPC分类号： B60W60/00274 ,  B60W60/0015 ,  B60W50/0097 ,  B60W40/04 ,  G06N3/049 ,  G06N3/08 ,  B60W2554/4029 ,  B60W2554/4045 ,  B60W2554/4046 ,  B60W2554/408 ,  B60W2556/10

摘要： Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent behavior prediction using keypoint data. One of the methods includes obtaining data characterizing a scene in an environment, the data comprising: (i) context data comprising data characterizing historical trajectories of a plurality of agents up to the current time point; and (ii) keypoint data for a target agent; processing the context data using a context data encoder neural network to generate a context embedding for the target agent; processing the keypoint data using a keypoint encoder neural network to generate a keypoint embedding for the target agent; generating a combined embedding for the target agent from the context embedding and the keypoint embedding; and processing the combined embedding using a decoder neural network to generate a behavior prediction output for the target agent that characterizes predicted behavior of the target agent after the current time point.