公开(公告)号：US11477243B2

公开(公告)日：2022-10-18

申请号：US16827596

申请日：2020-03-23

Applicant: Google LLC

Inventor： Kanury Kanishka Rao ,  Konstantinos Bousmalis ,  Christopher K. Harris ,  Alexander Irpan ,  Sergey Vladimir Levine ,  Julian Ibarz

IPC: G06N20/20 ,  H04L9/40 ,  G06K9/62 ,  G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for off-policy evaluation of a control policy. One of the methods includes obtaining policy data specifying a control policy for controlling a source agent interacting with a source environment to perform a particular task; obtaining a validation data set generated from interactions of a target agent in a target environment; determining a performance estimate that represents an estimate of a performance of the control policy in controlling the target agent to perform the particular task in the target environment; and determining, based on the performance estimate, whether to deploy the control policy for controlling the target agent to perform the particular task in the target environment.

公开(公告)号：US11717959B2

公开(公告)日：2023-08-08

申请号：US16622309

申请日：2018-06-28

Applicant: Google LLC

Inventor： Eric Jang ,  Sudheendra Vijayanarasimhan ,  Peter Pastor Sampedro ,  Julian Ibarz ,  Sergey Levine

IPC: B25J9/16 ,  G06N3/08 ,  G06N3/008 ,  G06N3/045

CPC classification number: B25J9/163 ,  G06N3/008 ,  G06N3/045 ,  G06N3/08 ,  G05B2219/39536

Abstract: Deep machine learning methods and apparatus related to semantic robotic grasping are provided. Some implementations relate to training a training a grasp neural network, a semantic neural network, and a joint neural network of a semantic grasping model. In some of those implementations, the joint network is a deep neural network and can be trained based on both: grasp losses generated based on grasp predictions generated over a grasp neural network, and semantic losses generated based on semantic predictions generated over the semantic neural network. Some implementations are directed to utilization of the trained semantic grasping model to servo, or control, a grasping end effector of a robot to achieve a successful grasp of an object having desired semantic feature(s).

公开(公告)号：US11341364B2

公开(公告)日：2022-05-24

申请号：US16649599

申请日：2018-09-20

Applicant: GOOGLE LLC

Inventor： Konstantinos Bousmalis ,  Alexander Irpan ,  Paul Wohlhart ,  Yunfei Bai ,  Mrinal Kalakrishnan ,  Julian Ibarz ,  Sergey Vladimir Levine ,  Kurt Konolige ,  Vincent O. Vanhoucke ,  Matthew Laurance Kelcey

IPC: G06K9/62 ,  B25J9/16 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network that is used to control a robotic agent interacting with a real-world environment.

公开(公告)号：US20200338722A1

公开(公告)日：2020-10-29

申请号：US16622309

申请日：2018-06-28

Applicant: Google LLC

Inventor： Eric Jang ,  Sudheendra Vijayanarasimhan ,  Peter Pastor Sampedro ,  Julian Ibarz ,  Sergey Levine

IPC: B25J9/16 ,  G06N3/04 ,  G06N3/08

Abstract: Deep machine learning methods and apparatus related to semantic robotic grasping are provided. Some implementations relate to training a training a grasp neural network, a semantic neural network, and a joint neural network of a semantic grasping model. In some of those implementations, the joint network is a deep neural network and can be trained based on both: grasp losses generated based on grasp predictions generated over a grasp neural network, and semantic losses generated based on semantic predictions generated over the semantic neural network. Some implementations are directed to utilization of the trained semantic grasping model to servo, or control, a grasping end effector of a robot to achieve a successful grasp of an object having desired semantic feature(s).

公开(公告)号：US20240189994A1

公开(公告)日：2024-06-13

申请号：US18539171

申请日：2023-12-13

Applicant: Google LLC

Inventor： Keerthana P G ,  Karol Hausman ,  Julian Ibarz ,  Brian Ichter ,  Alexander Irpan ,  Dmitry Kalashnikov ,  Yao Lu ,  Kanury Kanishka Rao ,  Michael Sahngwon Ryoo ,  Austin Charles Stone ,  Teddey Ming Xiao ,  Quan Ho Vuong ,  Sumedh Anand Sontakke

IPC: B25J9/16

CPC classification number: B25J9/163 ,  B25J9/161

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent interacting with an environment. In one aspect, a method comprises: receiving a natural language text sequence that characterizes a task to be performed by the agent in the environment; generating an encoded representation of the natural language text sequence; and at each of a plurality of time steps: obtaining an observation image characterizing a state of the environment at the time step; processing the observation image to generate an encoded representation of the observation image; generating a sequence of input tokens; processing the sequence of input tokens to generate a policy output that defines an action to be performed by the agent in response to the observation image; selecting an action to be performed by the agent using the policy output; and causing the agent to perform the selected action.

公开(公告)号：US20240118667A1

公开(公告)日：2024-04-11

申请号：US17767675

申请日：2020-05-15

Applicant: GOOGLE LLC

Inventor： Kanishka Rao ,  Chris Harris ,  Julian Ibarz ,  Alexander Irpan ,  Seyed Mohammad Khansari Zadeh ,  Sergey Levine

IPC: G05B13/02 ,  B25J9/16 ,  B25J19/02

CPC classification number: G05B13/0265 ,  B25J9/1605 ,  B25J9/163 ,  B25J9/1697 ,  B25J19/023

Abstract: Implementations disclosed herein relate to mitigating the reality gap through training a simulation-to-real machine learning model (“Sim2Real” model) using a vision-based robot task machine learning model. The vision-based robot task machine learning model can be, for example, a reinforcement learning (“RL”) neural network model (RL-network), such as an RL-network that represents a Q-function.

公开(公告)号：US20210237266A1

公开(公告)日：2021-08-05

申请号：US17052679

申请日：2019-06-14

Applicant: Google LLC

Inventor： Dmitry Kalashnikov ,  Alexander Irpan ,  Peter Pastor Sampedro ,  Julian Ibarz ,  Alexander Herzog ,  Eric Jang ,  Deirdre Quillen ,  Ethan Holly ,  Sergey Levine

IPC: B25J9/16 ,  G06N3/08

Abstract: Using large-scale reinforcement learning to train a policy model that can be utilized by a robot in performing a robotic task in which the robot interacts with one or more environmental objects. In various implementations, off-policy deep reinforcement learning is used to train the policy model, and the off-policy deep reinforcement learning is based on self-supervised data collection. The policy model can be a neural network model. Implementations of the reinforcement learning utilized in training the neural network model utilize a continuous-action variant of Q-learning. Through techniques disclosed herein, implementations can learn policies that generalize effectively to previously unseen objects, previously unseen environments, etc.

公开(公告)号：US20200304545A1

公开(公告)日：2020-09-24

申请号：US16827596

申请日：2020-03-23

Applicant: Google LLC

Inventor： Kanury Kanishka Rao ,  Konstantinos Bousmalis ,  Christopher K. Harris ,  Alexander Irpan ,  Sergey Vladimir Levine ,  Julian Ibarz

IPC: H04L29/06 ,  G06K9/62 ,  G06N3/04 ,  G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for off-policy evaluation of a control policy. One of the methods includes obtaining policy data specifying a control policy for controlling a source agent interacting with a source environment to perform a particular task; obtaining a validation data set generated from interactions of a target agent in a target environment; determining a performance estimate that represents an estimate of a performance of the control policy in controlling the target agent to perform the particular task in the target environment; and determining, based on the performance estimate, whether to deploy the control policy for controlling the target agent to perform the particular task in the target environment.

公开(公告)号：US20200279134A1

公开(公告)日：2020-09-03

申请号：US16649599

申请日：2018-09-20

Applicant: GOOGLE LLC

Inventor： Konstantinos Bousmalis ,  Alexander Irpan ,  Paul Wohlhart ,  Yunfei Bai ,  Mrinal Kalakrishnan ,  Julian Ibarz ,  Sergey Vladimir Levine ,  Kurt Konolige ,  Vincent O. Vanhoucke ,  Matthew Laurance Kelcey

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04 ,  B25J9/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network that is used to control a robotic agent interacting with a real-world environment.