公开(公告)号：US20240100693A1

公开(公告)日：2024-03-28

申请号：US18102053

申请日：2023-01-26

Applicant: GOOGLE LLC

Inventor： Daniel Ho ,  Eric Jang ,  Mohi Khansari ,  Yu Qing Du ,  Alexander A. Alemi

IPC: B25J9/16

CPC classification number: B25J9/163 ,  B25J9/1653 ,  B25J9/1697 ,  B25J9/162

Abstract: Some implementations relate to using trained robotic action ML models in controlling a robot to perform a robotic task. Some versions of those implementations include (a) a first modality robotic action ML model that is used to generate, based on processing first modality sensor data instances, first predicted action outputs for the robotic task and (b) a second modality robotic action ML model that is used to generate, in parallel and based on processing second modality sensor data instances, second predicted action outputs for the robotic task. In some of those versions, respective weights for each pair of the first and second predicted action outputs are dynamically determined based on analysis of embeddings generated in generating the first and second predicted action outputs. A final predicted action output, for controlling the robot, is determined based on the weights.

公开(公告)号：US11833661B2

公开(公告)日：2023-12-05

申请号：US17515490

申请日：2021-10-31

Applicant: GOOGLE LLC

Inventor： Zhuo Xu ,  Wenhao Yu ,  Alexander Herzog ,  Wenlong Lu ,  Chuyuan Fu ,  Yunfei Bai ,  C. Karen Liu ,  Daniel Ho

IPC: B25J9/16 ,  B25J13/08 ,  B25J19/02

CPC classification number: B25J9/163 ,  B25J9/161 ,  B25J9/1697 ,  B25J13/085 ,  B25J19/023

Abstract: Utilization of past dynamics sample(s), that reflect past contact physics information, in training and/or utilizing a neural network model. The neural network model represents a learned value function (e.g., a Q-value function) and that, when trained, can be used in selecting a sequence of robotic actions to implement in robotic manipulation (e.g., pushing) of an object by a robot. In various implementations, a past dynamics sample for an episode of robotic manipulation can include at least two past images from the episode, as well as one or more past force sensor readings that temporally correspond to the past images from the episode.