公开(公告)号：US10140198B1

公开(公告)日：2018-11-27

申请号：US13663959

申请日：2012-10-30

Applicant: Google LLC

Inventor： Priya Nayak ,  Eric Zhang ,  Jason J Ganetsky ,  James Watts ,  Robert B. Rose ,  Johnny Lee ,  Mike Sorvillo ,  Hubert Ming-Hwa Chao ,  Aruna Balakrishnan

IPC: G06F11/30 ,  H04N21/25

Abstract: A method and apparatus for a networked desktop environment is provided. Executing a networked desktop environment may include identifying a networked desktop state of the networked desktop environment, which may include a networked desktop workspace and an information element. The networked desktop state may include information indicating a multi-dimensional position of a representation of the information element in the networked desktop workspace. Executing the networked desktop environment may include generating instructions for use in rendering the networked desktop workspace, which may include information indicative of the networked desktop state, and transmitting the instructions to an interface device. Information indicating an interaction with the networked desktop environment may be received and the networked desktop state may be updated based on the interaction information.

公开(公告)号：US20220371195A1

公开(公告)日：2022-11-24

申请号：US17753979

申请日：2019-11-19

Applicant: Google LLC

Inventor： Johnny Lee ,  Stefan Welker

IPC: B25J9/16 ,  B25J13/08 ,  G06V20/50 ,  G05B19/4155

Abstract: An example computer-implemented method includes receiving, from one or more vision components in an environment, vision data that captures features of the environment, including object features of an object that is located in the environment, and prior to a robot manipulating the object: (i) determining based on the vision data, at least one first adjustment to a programmed trajectory of movement of the robot operating in the environment to perform a task of transporting the object, and (ii) determining based on the object features of the object, at least one second adjustment to the programmed trajectory of movement of the robot operating in the environment to perform the task, and causing the robot to perform the task, in accordance with the at least one first adjustment and the at least one second adjustment to the programmed trajectory of movement of the robot.

公开(公告)号：US10852847B2

公开(公告)日：2020-12-01

申请号：US15660216

申请日：2017-07-26

Applicant: GOOGLE LLC

Inventor： Joel Hesch ,  Shiqi Chen ,  Johnny Lee ,  Rahul Garg

IPC: G06F3/0346 ,  G06T7/73 ,  G06F3/0354 ,  G06F3/01

Abstract: A method for controller tracking with multiple degrees of freedom includes generating depth data at an electronic device based on a local environment proximate the electronic device. A set of positional data is generated for at least one spatial feature associated with a controller based on a pose of the electronic device, as determined using the depth data, relative to the at least one spatial feature associated with the controller. A set of rotational data is received that represents three degrees-of-freedom (3DoF) orientation of the controller within the local environment, and a six degrees-of-freedom (6DoF) position of the controller within the local environment is tracked based on the set of positional data and the set of rotational data.

公开(公告)号：US20230398690A1

公开(公告)日：2023-12-14

申请号：US18233261

申请日：2023-08-11

Applicant: GOOGLE LLC

Inventor： Johnny Lee ,  Stefan Welker

IPC: B25J9/16

CPC classification number: B25J9/1689 ,  B25J9/163 ,  B25J9/1697 ,  B25J9/1612

Abstract: Utilization of user interface inputs, from remote client devices, in controlling robot(s) in an environment. Implementations relate to generating training instances based on object manipulation parameters, defined by instances of user interface input(s), and training machine learning model(s) to predict the object manipulation parameter(s). Those implementations can subsequently utilize the trained machine learning model(s) to reduce a quantity of instances that input(s) from remote client device(s) are solicited in performing a given set of robotic manipulations and/or to reduce the extent of input(s) from remote client device(s) in performing a given set of robotic operations. Implementations are additionally or alternatively related to mitigating idle time of robot(s) through the utilization of vision data that captures object(s), to be manipulated by a robot, prior to the object(s) being transported to a robot workspace within which the robot can reach and manipulate the object.

公开(公告)号：US11724398B2

公开(公告)日：2023-08-15

申请号：US17535393

申请日：2021-11-24

Applicant: Google LLC

Inventor： Johnny Lee ,  Stefan Welker

IPC: B25J9/16

CPC classification number: B25J9/1689 ,  B25J9/163 ,  B25J9/1697 ,  B25J9/1612

Abstract: Utilization of user interface inputs, from remote client devices, in controlling robot(s) in an environment. Implementations relate to generating training instances based on object manipulation parameters, defined by instances of user interface input(s), and training machine learning model(s) to predict the object manipulation parameter(s). Those implementations can subsequently utilize the trained machine learning model(s) to reduce a quantity of instances that input(s) from remote client device(s) are solicited in performing a given set of robotic manipulations and/or to reduce the extent of input(s) from remote client device(s) in performing a given set of robotic operations. Implementations are additionally or alternatively related to mitigating idle time of robot(s) through the utilization of vision data that captures object(s), to be manipulated by a robot, prior to the object(s) being transported to a robot workspace within which the robot can reach and manipulate the object.

公开(公告)号：US11213953B2

公开(公告)日：2022-01-04

申请号：US16523655

申请日：2019-07-26

Applicant: Google LLC

Inventor： Johnny Lee ,  Stefan Welker

IPC: B25J9/16

Abstract: Utilization of user interface inputs, from remote client devices, in controlling robot(s) in an environment. Implementations relate to generating training instances based on object manipulation parameters, defined by instances of user interface input(s), and training machine learning model(s) to predict the object manipulation parameter(s). Those implementations can subsequently utilize the trained machine learning model(s) to reduce a quantity of instances that input(s) from remote client device(s) are solicited in performing a given set of robotic manipulations and/or to reduce the extent of input(s) from remote client device(s) in performing a given set of robotic operations. Implementations are additionally or alternatively related to mitigating idle time of robot(s) through the utilization of vision data that captures object(s), to be manipulated by a robot, prior to the object(s) being transported to a robot workspace within which the robot can reach and manipulate the object.

公开(公告)号：US12138810B2

公开(公告)日：2024-11-12

申请号：US18233261

申请日：2023-08-11

Applicant: GOOGLE LLC

Inventor： Johnny Lee ,  Stefan Welker

IPC: B25J9/16

Abstract: Utilization of user interface inputs, from remote client devices, in controlling robot(s) in an environment. Implementations relate to generating training instances based on object manipulation parameters, defined by instances of user interface input(s), and training machine learning model(s) to predict the object manipulation parameter(s). Those implementations can subsequently utilize the trained machine learning model(s) to reduce a quantity of instances that input(s) from remote client device(s) are solicited in performing a given set of robotic manipulations and/or to reduce the extent of input(s) from remote client device(s) in performing a given set of robotic operations. Implementations are additionally or alternatively related to mitigating idle time of robot(s) through the utilization of vision data that captures object(s), to be manipulated by a robot, prior to the object(s) being transported to a robot workspace within which the robot can reach and manipulate the object.

公开(公告)号：US20220355483A1

公开(公告)日：2022-11-10

申请号：US17753978

申请日：2019-11-19

Applicant: Google LLC

Inventor： Johnny Lee ,  Stefan Welker

IPC: B25J9/16 ,  B25J13/02 ,  B25J13/08

Abstract: An example method for providing a graphical user interface (GUI) of a computing device includes receiving an input indicating a target pose of the robot, providing for display on the GUI of the computing device a transparent representation of the robot as a preview of the target pose in combination with the textured model of the robot indicating the current state of the robot, generating a boundary illustration on the GUI representative of a limit of a range of motion of the robot, based on the target pose extending the robot beyond the boundary illustration, modifying characteristics of the transparent representation of the robot and of the boundary illustration on the GUI to inform of an invalid pose, and based on the target pose being a valid pose, sending instructions to the robot causing the robot to perform the target pose.

公开(公告)号：US20220152833A1

公开(公告)日：2022-05-19

申请号：US17535393

申请日：2021-11-24

Applicant: Google LLC

Inventor： Johnny Lee ,  Stefan Welker

IPC: B25J9/16

Abstract: Utilization of user interface inputs, from remote client devices, in controlling robot(s) in an environment. Implementations relate to generating training instances based on object manipulation parameters, defined by instances of user interface input(s), and training machine learning model(s) to predict the object manipulation parameter(s). Those implementations can subsequently utilize the trained machine learning model(s) to reduce a quantity of instances that input(s) from remote client device(s) are solicited in performing a given set of robotic manipulations and/or to reduce the extent of input(s) from remote client device(s) in performing a given set of robotic operations. Implementations are additionally or alternatively related to mitigating idle time of robot(s) through the utilization of vision data that captures object(s), to be manipulated by a robot, prior to the object(s) being transported to a robot workspace within which the robot can reach and manipulate the object.

公开(公告)号：US20210023711A1

公开(公告)日：2021-01-28

申请号：US16523655

申请日：2019-07-26

Applicant: Google LLC

Inventor： Johnny Lee ,  Stefan Welker

IPC: B25J9/16

Abstract: Utilization of user interface inputs, from remote client devices, in controlling robot(s) in an environment. Implementations relate to generating training instances based on object manipulation parameters, defined by instances of user interface input(s), and training machine learning model(s) to predict the object manipulation parameter(s). Those implementations can subsequently utilize the trained machine learning model(s) to reduce a quantity of instances that input(s) from remote client device(s) are solicited in performing a given set of robotic manipulations and/or to reduce the extent of input(s) from remote client device(s) in performing a given set of robotic operations. Implementations are additionally or alternatively related to mitigating idle time of robot(s) through the utilization of vision data that captures object(s), to be manipulated by a robot, prior to the object(s) being transported to a robot workspace within which the robot can reach and manipulate the object.