公开(公告)号：US20240249483A1

公开(公告)日：2024-07-25

申请号：US18623987

申请日：2024-04-01

Applicant: Snap Inc.

Inventor： Kai Zhou ,  Adrian Schoisengeier

IPC: G06T19/00 ,  B25J13/08 ,  G06F3/01 ,  G06T17/00 ,  G06T19/20 ,  G06V20/64

CPC classification number: G06T19/006 ,  B25J13/08 ,  G06F3/012 ,  G06F3/017 ,  G06T17/00 ,  G06T19/20 ,  G06V20/64 ,  G06T2200/08 ,  G06T2210/56 ,  G06T2219/2016

Abstract: A method for programming a robotic system by demonstration is described. In one aspect, the method includes displaying a first virtual object in a display of an augmented reality (AR) device, the first virtual object corresponding to a first physical object in a physical environment of the AR device, tracking, using the AR device, a manipulation of the first virtual object by a user of the AR device, identifying an initial state and a final state of the first virtual object based on the tracking, the initial state corresponding to an initial pose of the first virtual object, the final state corresponding to a final pose of the first virtual object, and programming by demonstration a robotic system using the tracking of the manipulation of the first virtual object, the first initial state of the first virtual object, and the final state of the first virtual object.

公开(公告)号：US11989843B2

公开(公告)日：2024-05-21

申请号：US17846930

申请日：2022-06-22

Applicant: Snap Inc.

Inventor： Kai Zhou ,  Adrian Schoisengeier

IPC: G06T19/00 ,  B25J13/08 ,  G06F3/01 ,  G06T17/00 ,  G06T19/20 ,  G06V20/64 ,  G06V40/10

CPC classification number: G06T19/006 ,  B25J13/08 ,  G06F3/012 ,  G06F3/017 ,  G06T17/00 ,  G06T19/20 ,  G06V20/64 ,  G06T2200/08 ,  G06T2210/56 ,  G06T2219/2016

Abstract: A method for programming a robotic system by demonstration is described. In one aspect, the method includes displaying a first virtual object in a display of an augmented reality (AR) device, the first virtual object corresponding to a first physical object in a physical environment of the AR device, tracking, using the AR device, a manipulation of the first virtual object by a user of the AR device, identifying an initial state and a final state of the first virtual object based on the tracking, the initial state corresponding to an initial pose of the first virtual object, the final state corresponding to a final pose of the first virtual object, and programming by demonstration a robotic system using the tracking of the manipulation of the first virtual object, the first initial state of the first virtual object, and the final state of the first virtual object.

公开(公告)号：US20250061658A1

公开(公告)日：2025-02-20

申请号：US18234737

申请日：2023-08-16

Applicant: Snap Inc.

Inventor： Laura Rosalia Luidolt ,  Kai Zhou ,  Adrian Schoisengeier

IPC: G06T17/20 ,  G06T7/70 ,  G06T13/40 ,  G06T15/20

Abstract: An augmented reality (AR) device generates hand annotations for an image depicting a user's hand. The device includes a display, a processor, and a memory storing instructions for performing operations. The device performs a calibration operation to generate a 3-D model of the user's hand based on measurements. The calibration operation prompts the user to mimic the hand gesture presented by the animated virtual representation of a hand to obtain optimal images of the hand for hand pose estimation. A 3-D virtual representation of a hand in a hand pose corresponding with a hand gesture is generated based on the 3-D model. The device presents the 3-D virtual representation of the hand in AR via the display. During presentation, the device detects an input and captures an image of the user's hand positioned to correspond with the 3-D virtual representation. The captured image is stored with corresponding hand annotations based on the 3-D model.

公开(公告)号：US20230419615A1

公开(公告)日：2023-12-28

申请号：US17846930

申请日：2022-06-22

Applicant: Snap Inc.

Inventor： Kai Zhou ,  Adrian Schoisengeier

IPC: G06T19/00 ,  G06V20/64 ,  G06F3/01 ,  G06T17/00 ,  G06T19/20 ,  B25J13/08

CPC classification number: G06T19/006 ,  G06V20/64 ,  G06F3/017 ,  G06T17/00 ,  G06T19/20 ,  G06F3/012 ,  B25J13/08 ,  G06T2200/08 ,  G06T2210/56 ,  G06T2219/2016

Abstract: A method for programming a robotic system by demonstration is described. In one aspect, the method includes displaying a first virtual object in a display of an augmented reality (AR) device, the first virtual object corresponding to a first physical object in a physical environment of the AR device, tracking, using the AR device, a manipulation of the first virtual object by a user of the AR device, identifying an initial state and a final state of the first virtual object based on the tracking, the initial state corresponding to an initial pose of the first virtual object, the final state corresponding to a final pose of the first virtual object, and programming by demonstration a robotic system using the tracking of the manipulation of the first virtual object, the first initial state of the first virtual object, and the final state of the first virtual object.