公开(公告)号：US12153724B2

公开(公告)日：2024-11-26

申请号：US17719630

申请日：2022-04-13

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Tsz Ho Yu ,  Chengyuan Yan ,  Christian Forster

IPC: G06F3/01 ,  G06F18/25 ,  G06V20/10 ,  G06V40/20

Abstract: In one embodiment, a method includes capturing, using one or more cameras implemented in a wearable device worn by a user, a first image depicting at least a part of a hand of the user holding a controller in an environment, identifying one or more features from the first image to estimate a pose of the hand of the user, estimating a first pose of the controller based on the pose of the hand of the user and an estimated grip that defines a relative pose between the hand of the user and the controller, receiving IMU data of the controller, and estimating a second pose of the controller by updating the first pose of the controller using the IMU data of the controller. The method utilizes multiple data sources to track the controller under various conditions of the environment to provide an accurate controller tracking consistently.

公开(公告)号：US20250123679A1

公开(公告)日：2025-04-17

申请号：US18920449

申请日：2024-10-18

Applicant: Meta Platforms Technologies, LLC

Inventor： Tsz Ho Yu ,  Chengyuan Yan ,  Christian Forster

IPC: G06F3/01 ,  G06F18/25 ,  G06V20/10 ,  G06V40/20

Abstract: In one embodiment, a method includes capturing, using one or more cameras implemented in a wearable device worn by a user, a first image depicting at least a part of a hand of the user holding a controller in an environment, identifying one or more features from the first image to estimate a pose of the hand of the user, estimating a first pose of the controller based on the pose of the hand of the user and an estimated grip that defines a relative pose between the hand of the user and the controller, receiving IMU data of the controller, and estimating a second pose of the controller by updating the first pose of the controller using the IMU data of the controller. The method utilizes multiple data sources to track the controller under various conditions of the environment to provide an accurate controller tracking consistently.