公开(公告)号：US20250131335A1

公开(公告)日：2025-04-24

申请号：US18991973

申请日：2024-12-23

Applicant: GOOGLE LLC

Inventor： Adrian Li ,  Benjamin Holson ,  Alexander Herzog ,  Mrinal Kalakrishnan

IPC: G06N20/00 ,  G06N3/008 ,  G06N5/04

Abstract: Implementations disclosed herein relate to utilizing at least one existing manually engineered policy, for a robotic task, in training an RL policy model that can be used to at least selectively replace a portion of the engineered policy. The RL policy model can be trained for replacing a portion of a robotic task and can be trained based on data from episodes of attempting performance of the robotic task, including episodes in which the portion is performed based on the engineered policy and/or other portion(s) are performed based on the engineered policy. Once trained, the RL policy model can be used, at least selectively and in lieu of utilization of the engineered policy, to perform the portion of robotic task, while other portion(s) of the robotic task are performed utilizing the engineered policy and/or other similarly trained (but distinct) RL policy model(s).

公开(公告)号：US20250033201A1

公开(公告)日：2025-01-30

申请号：US18918590

申请日：2024-10-17

Applicant: GOOGLE LLC

Inventor： Yunfei Bai ,  Kuan Fang ,  Stefan Hinterstoisser ,  Mrinal Kalakrishnan

IPC: B25J9/16

Abstract: Implementations are directed to training a machine learning model that, once trained, is used in performance of robotic grasping and/or other manipulation task(s) by a robot. The model can be trained using simulated training examples that are based on simulated data that is based on simulated robot(s) attempting simulated manipulations of various simulated objects. Portion(s) of the model can also be trained based on real training examples that are based on data from real-world physical robots attempting manipulations of various objects. The simulated training examples can be utilized to train the model to predict an output that can be utilized in a particular task—and the real training examples used to adapt at least a portion of the model to the real-world domain can be tailored to a distinct task. In some implementations, domain-adversarial similarity losses are determined during training, and utilized to regularize at least portion(s) of the model.

公开(公告)号：US12083678B2

公开(公告)日：2024-09-10

申请号：US17422260

申请日：2020-01-23

Applicant: Google LLC

Inventor： Mrinal Kalakrishnan ,  Yunfei Bai ,  Paul Wohlhart ,  Eric Jang ,  Chelsea Finn ,  Seyed Mohammad Khansari Zadeh ,  Sergey Levine ,  Allan Zhou ,  Alexander Herzog ,  Daniel Kappler

IPC: B25J9/16

CPC classification number: B25J9/163 ,  G05B2219/40116 ,  G05B2219/40499

Abstract: Techniques are disclosed that enable training a meta-learning model, for use in causing a robot to perform a task, using imitation learning as well as reinforcement learning. Some implementations relate to training the meta-learning model using imitation learning based on one or more human guided demonstrations of the task. Additional or alternative implementations relate to training the meta-learning model using reinforcement learning based on trials of the robot attempting to perform the task. Further implementations relate to using the trained meta-learning model to few shot (or one shot) learn a new task based on a human guided demonstration of the new task.

公开(公告)号：US12210943B2

公开(公告)日：2025-01-28

申请号：US17161845

申请日：2021-01-29

Applicant: GOOGLE LLC

Inventor： Adrian Li ,  Benjamin Holson ,  Alexander Herzog ,  Mrinal Kalakrishnan

IPC: G06N20/00 ,  G06N3/008 ,  G06N5/04

Abstract: Implementations disclosed herein relate to utilizing at least one existing manually engineered policy, for a robotic task, in training an RL policy model that can be used to at least selectively replace a portion of the engineered policy. The RL policy model can be trained for replacing a portion of a robotic task and can be trained based on data from episodes of attempting performance of the robotic task, including episodes in which the portion is performed based on the engineered policy and/or other portion(s) are performed based on the engineered policy. Once trained, the RL policy model can be used, at least selectively and in lieu of utilization of the engineered policy, to perform the portion of robotic task, while other portion(s) of the robotic task are performed utilizing the engineered policy and/or other similarly trained (but distinct) RL policy model(s).

公开(公告)号：US20240238967A1

公开(公告)日：2024-07-18

申请号：US18605452

申请日：2024-03-14

Applicant: Google LLC

Inventor： Paul Wohlhart ,  Stephen James ,  Mrinal Kalakrishnan ,  Konstantinos Bousmalis

IPC: B25J9/16 ,  G05B13/02 ,  G06F18/21 ,  G06F18/214 ,  G06F18/2431 ,  G06N3/045 ,  G06N3/08 ,  G06T7/50 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V20/10

CPC classification number: B25J9/161 ,  B25J9/163 ,  B25J9/1671 ,  B25J9/1697 ,  G05B13/027 ,  G06F18/2148 ,  G06F18/217 ,  G06F18/2431 ,  G06N3/045 ,  G06N3/08 ,  G06T7/50 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V20/10 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a generator neural network to adapt input images.

公开(公告)号：US11951622B2

公开(公告)日：2024-04-09

申请号：US17656137

申请日：2022-03-23

Applicant: Google LLC

Inventor： Paul Wohlhart ,  Stephen James ,  Mrinal Kalakrishnan ,  Konstantinos Bousmalis

IPC: G06T7/00 ,  B25J9/16 ,  G05B13/02 ,  G06F18/21 ,  G06F18/214 ,  G06F18/2431 ,  G06N3/045 ,  G06N3/08 ,  G06T7/50 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V20/10

CPC classification number: B25J9/161 ,  B25J9/163 ,  B25J9/1671 ,  B25J9/1697 ,  G05B13/027 ,  G06F18/2148 ,  G06F18/217 ,  G06F18/2431 ,  G06N3/045 ,  G06N3/08 ,  G06T7/50 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V20/10 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a generator neural network to adapt input images.

公开(公告)号：US11341364B2

公开(公告)日：2022-05-24

申请号：US16649599

申请日：2018-09-20

Applicant: GOOGLE LLC

Inventor： Konstantinos Bousmalis ,  Alexander Irpan ,  Paul Wohlhart ,  Yunfei Bai ,  Mrinal Kalakrishnan ,  Julian Ibarz ,  Sergey Vladimir Levine ,  Kurt Konolige ,  Vincent O. Vanhoucke ,  Matthew Laurance Kelcey

IPC: G06K9/62 ,  B25J9/16 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network that is used to control a robotic agent interacting with a real-world environment.

公开(公告)号：US12138793B2

公开(公告)日：2024-11-12

申请号：US16987771

申请日：2020-08-07

Applicant: GOOGLE LLC

Inventor： Yunfei Bai ,  Kuan Fang ,  Stefan Hinterstoisser ,  Mrinal Kalakrishnan

IPC: B25J9/16

Abstract: Implementations are directed to training a machine learning model that, once trained, is used in performance of robotic grasping and/or other manipulation task(s) by a robot. The model can be trained using simulated training examples that are based on simulated data that is based on simulated robot(s) attempting simulated manipulations of various simulated objects. Portion(s) of the model can also be trained based on real training examples that are based on data from real-world physical robots attempting manipulations of various objects. The simulated training examples can be utilized to train the model to predict an output that can be utilized in a particular task—and the real training examples used to adapt at least a portion of the model to the real-world domain can be tailored to a distinct task. In some implementations, domain-adversarial similarity losses are determined during training, and utilized to regularize at least portion(s) of the model.

公开(公告)号：US20220105624A1

公开(公告)日：2022-04-07

申请号：US17422260

申请日：2020-01-23

Applicant: Google LLC

Inventor： Mrinal Kalakrishnan ,  Yunfei Bai ,  Paul Wohlhart ,  Eric Jang ,  Chelsea Finn ,  Seyed Mohammad Khansari Zadeh ,  Sergey Levine ,  Allan Zhou ,  Alexander Herzog ,  Daniel Kappler

IPC: B25J9/16

Abstract: Techniques are disclosed that enable training a meta-learning model, for use in causing a robot to perform a task, using imitation learning as well as reinforcement learning. Some implementations relate to training the meta-learning model using imitation learning based on one or more human guided demonstrations of the task. Additional or alternative implementations relate to training the meta-learning model using reinforcement learning based on trials of the robot attempting to perform the task. Further implementations relate to using the trained meta-learning model to few shot (or one shot) learn a new task based on a human guided demonstration of the new task.

公开(公告)号：US12049004B1

公开(公告)日：2024-07-30

申请号：US18103312

申请日：2023-01-30

Applicant: GOOGLE LLC

Inventor： Peter Pastor Sampedro ,  Mrinal Kalakrishnan ,  Ali Yahya Valdovinos ,  Adrian Li ,  Kurt Konolige ,  Vincent Dureau

IPC: B25J9/00 ,  B25J9/16

CPC classification number: B25J9/0084 ,  B25J9/163 ,  G05B2219/39271

Abstract: Methods and apparatus related to receiving a request that includes robot instructions and/or environmental parameters, operating each of a plurality of robots based on the robot instructions and/or in an environment configured based on the environmental parameters, and storing data generated by the robots during the operating. In some implementations, at least part of the stored data that is generated by the robots is provided in response to the request and/or additional data that is generated based on the stored data is provided in response to the request.