公开(公告)号：US12056275B2

公开(公告)日：2024-08-06

申请号：US18048411

申请日：2022-10-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Daniel Miller ,  Renzo De Nardi ,  Richard Andrew Newcombe

IPC: G06F3/01 ,  G02B27/00 ,  G02B27/01 ,  G06N20/00 ,  G06V10/25 ,  G06V20/20 ,  G06V40/18

CPC classification number: G06F3/013 ,  G02B27/0093 ,  G02B27/0101 ,  G02B27/017 ,  G06N20/00 ,  G06V10/25 ,  G06V20/20 ,  G06V40/18 ,  G02B2027/014 ,  G02B2027/0178

Abstract: An Artificial Reality (AR) device captures an image of a physical environment surrounding a user wearing the AR device. A three-dimensional map corresponding to the physical environment is accessed. Then, a pose of the AR device relative to the three-dimensional map is determined based on first features of physical objects captured in the image and second features of object representations in the three-dimensional map. The device determines, using an eye tracker, a gaze of an eye of the user. Based on the gaze and the pose, a region of interest is computed in the three-dimensional map. Representations of physical devices in the region of interest of the three-dimensional map are identified. The device determines an intent of the user to interact with a physical device. A command is issued to the physical device based on the determined intent.

公开(公告)号：US20240411364A1

公开(公告)日：2024-12-12

申请号：US18789518

申请日：2024-07-30

Applicant: Meta Platforms Technologies, LLC

Inventor： Daniel Miller ,  Renzo De Nardi ,  Richard Andrew Newcombe

IPC: G06F3/01 ,  G02B27/00 ,  G02B27/01 ,  G06N20/00 ,  G06V10/25 ,  G06V20/20 ,  G06V40/18

Abstract: A non-transitory, computer-readable storage medium including executable instructions that, when executed by one or more processors, allow a user to interact with physical devices via an artificial-reality (AR) device is described. In response to a command received from the user of a head-wearable device, the instructions cause the one or more processors to (i) obtain an image of a physical environment surrounding the user wearing the head-wearable device, (ii) determine a region of interest, including one or more available physical devices, in the image based at least on a gaze of an eye of the user, (iii) determine, by a machine-learning model, an intent of the user to interact with a particular physical device of the one or more available physical devices, and (iv), based on the intent of the user to interact with a particular physical device, send a representation of the command to the particular physical device.

公开(公告)号：US20230130770A1

公开(公告)日：2023-04-27

申请号：US18048411

申请日：2022-10-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Daniel Miller ,  Renzo De Nardi ,  Richard Andrew Newcombe

IPC: G06F3/01 ,  G02B27/00 ,  G02B27/01 ,  G06V40/18 ,  G06V10/25 ,  G06V20/20 ,  G06N20/00

Abstract: An Artificial Reality (AR) device captures an image of a physical environment surrounding a user wearing the AR device. A three-dimensional map corresponding to the physical environment is accessed. Then, a pose of the AR device relative to the three-dimensional map is determined based on first features of physical objects captured in the image and second features of object representations in the three-dimensional map. The device determines, using an eye tracker, a gaze of an eye of the user. Based on the gaze and the pose, a region of interest is computed in the three-dimensional map. Representations of physical devices in the region of interest of the three-dimensional map are identified. The device determines an intent of the user to interact with a physical device. A command is issued to the physical device based on the determined intent.