公开(公告)号：US11915523B2

公开(公告)日：2024-02-27

申请号：US17815361

申请日：2022-07-27

Applicant: Google LLC

Inventor： Sean Kirmani ,  Michael Quinlan ,  Sarah Coe

IPC: G06V40/20 ,  B25J9/16 ,  G05B13/02 ,  G06V10/40 ,  G06V20/10 ,  G06V40/10

CPC classification number: G06V40/23 ,  B25J9/163 ,  B25J9/1697 ,  G05B13/0265 ,  G06V10/40 ,  G06V20/10 ,  G06V40/103

Abstract: A method includes receiving, from a camera disposed on a robotic device, a two-dimensional (2D) image of a body of an actor and determining, for each respective keypoint of a first subset of a plurality of keypoints, 2D coordinates of the respective keypoint within the 2D image. The plurality of keypoints represent body locations. Each respective keypoint of the first subset is visible in the 2D image. The method also includes determining a second subset of the plurality of keypoints. Each respective keypoint of the second subset is not visible in the 2D image. The method further includes determining, by way of a machine learning model, an extent of engagement of the actor with the robotic device based on (i) the 2D coordinates of keypoints of the first subset and (ii) for each respective keypoint of the second subset, an indicator that the respective keypoint is not visible.

公开(公告)号：US20240153314A1

公开(公告)日：2024-05-09

申请号：US18417414

申请日：2024-01-19

Applicant: Google LLC

Inventor： Sean Kirmani ,  Michael Quinlan ,  Sarah Coe

IPC: G06V40/20 ,  B25J9/16 ,  G05B13/02 ,  G06V10/40 ,  G06V20/10 ,  G06V40/10

CPC classification number: G06V40/23 ,  B25J9/163 ,  B25J9/1697 ,  G05B13/0265 ,  G06V10/40 ,  G06V20/10 ,  G06V40/103

Abstract: A method includes receiving, from a camera disposed on a robotic device, a two-dimensional (2D) image of a body of an actor and determining, for each respective keypoint of a first subset of a plurality of keypoints, 2D coordinates of the respective keypoint within the 2D image. The plurality of keypoints represent body locations. Each respective keypoint of the first subset is visible in the 2D image. The method also includes determining a second subset of the plurality of keypoints. Each respective keypoint of the second subset is not visible in the 2D image. The method further includes determining, by way of a machine learning model, an extent of engagement of the actor with the robotic device based on (i) the 2D coordinates of keypoints of the first subset and (ii) for each respective keypoint of the second subset, an indicator that the respective keypoint is not visible.