公开(公告)号：US20230131667A1

公开(公告)日：2023-04-27

申请号：US18069164

申请日：2022-12-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Adrian Brian Ratter ,  Alessia Marra ,  Yugeng He ,  Panya Inversin

IPC: G06F3/04815 ,  G06T7/70 ,  G02B27/00 ,  G02B27/01 ,  G06T7/10

Abstract: A method includes accessing an image of a physical environment of a user, the image depicting a physical input device and a physical hand of the user, determining that a contrast between the physical input device and the physical hand depicted in the image is lower than a predetermined threshold, modifying the image to increase the contrast, determining a pose of the physical input device, generating a three-dimensional model representing the physical hand of the user, generating an image mask by projecting the three-dimensional model onto an image plane, generating, a cropped image depicting at least the physical hand of the user in the image, rendering, based on the perspective of the user and the pose of the physical input device, a virtual input device to represent the physical input device, and displaying the cropped image depicting at least the physical hand over the rendered virtual input device.

公开(公告)号：US11914836B2

公开(公告)日：2024-02-27

申请号：US18069164

申请日：2022-12-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Adrian Brian Ratter ,  Alessia Marra ,  Yugeng He ,  Panya Inversin

IPC: G06F3/04815 ,  G02B27/00 ,  G02B27/01 ,  G06T7/10 ,  G06T7/70

CPC classification number: G06F3/04815 ,  G02B27/0093 ,  G02B27/017 ,  G06T7/10 ,  G06T7/70 ,  G06T2207/20132

Abstract: A method includes accessing an image of a physical environment of a user, the image depicting a physical input device and a physical hand of the user, determining that a contrast between the physical input device and the physical hand depicted in the image is lower than a predetermined threshold, modifying the image to increase the contrast, determining a pose of the physical input device, generating a three-dimensional model representing the physical hand of the user, generating an image mask by projecting the three-dimensional model onto an image plane, generating, a cropped image depicting at least the physical hand of the user in the image, rendering, based on the perspective of the user and the pose of the physical input device, a virtual input device to represent the physical input device, and displaying the cropped image depicting at least the physical hand over the rendered virtual input device.

公开(公告)号：US11537258B2

公开(公告)日：2022-12-27

申请号：US17073200

申请日：2020-10-16

Applicant: Meta Platforms Technologies, LLC

Inventor： Adrian Brian Ratter ,  Alessia Marra ,  Yugeng He ,  Panya Inversin

IPC: G06F3/04815 ,  G06T7/70 ,  G02B27/00 ,  G02B27/01 ,  G06T7/10

Abstract: In one embodiment, a method includes a computer system accessing an image of a physical environment of a user, the image being associated with a perspective of the user and depicting a physical input device and a physical hand of the user, determining a pose of the physical input device, generating a three-dimensional model representing the physical hand of the user, generating an image mask by projecting the three-dimensional model onto an image plane associated with the perspective of the user, generating, by applying the image mask to the image, a cropped image depicting at least the physical hand of the user in the image, rendering, based on the perspective of the user and the pose of the physical input device, a virtual input device to represent the physical input device, and displaying the cropped image depicting at least the physical hand over the rendered virtual input device.

公开(公告)号：US11500413B2

公开(公告)日：2022-11-15

申请号：US17007401

申请日：2020-08-31

Applicant: Meta Platforms Technologies, LLC

Inventor： Boyang Zhang ,  Adrian Brian Ratter ,  Chengyuan Yan ,  Jack Hood Profit, Jr. ,  Paul Austin Buckley

IPC: G06F1/12 ,  H04W4/80 ,  G06T19/00 ,  H04L43/0829 ,  H04L67/131

Abstract: In one embodiment, a method includes receiving, from a controller, a data packet including (1) a plurality of samples each corresponding to measurements from a motion sensor and (2) a timestamp corresponding to a measurement time of one of the samples as measured by a clock of the controller; determining, based on the timestamp, an estimated measurement time relative to a local clock for each of the plurality of samples that is not associated with the timestamp; and converting each of the timestamp and the estimated measurement times to a corresponding synchronization time using a learned relationship relating the clock of the controller and the local clock. The learned relationship is iteratively learned based on previously received data packets from the controller. The synchronization time associated with each of the plurality of samples represents an estimated time, relative to the local clock, at which the sample was measured.