公开(公告)号：US20250054176A1

公开(公告)日：2025-02-13

申请号：US18475720

申请日：2023-09-27

Applicant: Snap Inc.

Inventor： Evangelos Chatzikalymnios ,  Thomas Faeulhammer ,  Ihor Tymchyshyn ,  Daniel Wolf

IPC: G06T7/70 ,  G06T3/40 ,  G06T7/20 ,  G06T7/62 ,  G06V10/25 ,  G06V40/20

Abstract: A device includes a processor, image sensors, and memory storing instructions to obtain images from the sensors and process a first image to identify coordinates of a bounding box around an object. The device processes the area within the first box to determine 2-D positions of landmarks associated with the object, derives first 3-D positions of the landmarks, and determines coordinates of a second box bounding the object in the second image using the 3-D landmark positions. The device processes the area within the second box to determine 2-D positions of landmarks and uses triangulation to derive second 3-D positions of the landmarks. Overall, the device obtains images, detects objects and landmarks, determines 2-D and 3-D positions of landmarks, and triangulates 3-D positions.

公开(公告)号：US20240265658A1

公开(公告)日：2024-08-08

申请号：US18298217

申请日：2023-04-10

Applicant: Snap Inc.

Inventor： Maksym Bekuzarov ,  Andrii Didkivskyi ,  Sergei Korolev ,  Ihor Tymchyshyn

IPC: G06T19/20 ,  G06T19/00

CPC classification number: G06T19/20 ,  G06T19/006 ,  G06T2210/12 ,  G06T2210/22 ,  G06T2219/2012 ,  G06T2219/2021

Abstract: An extended Reality (XR) system provides methodologies for displaying virtual objects in a hand-centric XR experience. The XR system provides an XR user interface of an XR system to a user. The XR system captures video frame data of a hand of the user and detects the hand of the user based on the video frame data and a hand-detecting model. The XR system generates a cropping boundary box based on the detection of the hand and the video frame data and generates cropped video frame data based on the cropping boundary box and the video frame data. The XR system generates a 3D model of a portion of the hand of the user based on the cropped video frame data and a virtual object based on the 3D model of the portion of the hand of the user and a 3D texture. The XR displays the virtual object in the XR user interface.