公开(公告)号：US20230182296A1

公开(公告)日：2023-06-15

申请号：US17924891

申请日：2021-05-14

Applicant: GOOGLE LLC

Inventor： Pierre Sermanet ,  Corey Lynch

IPC: B25J9/16

CPC classification number: B25J9/1664 ,  B25J9/163 ,  B25J9/1697

Abstract: Techniques are disclosed that enable training a goal-conditioned policy based on multiple data sets, where each of the data sets describes a robot task in a different way. For example, the multiple data sets can include: a goal image data set, where the task is captured in the goal image; a natural language instruction data set, where the task is described in the natural language instruction; a task ID data set, where the task is described by the task ID, etc. In various implementations, each of the multiple data sets has a corresponding encoder, where the encoders are trained to generate a shared latent space representation of the corresponding task description. Additional or alternative techniques are disclosed that enable control of a robot using a goal-conditioned policy network. For example, the robot can be controlled, using the goal-conditioned policy network, based on free-form natural language input describing robot task(s).