公开(公告)号：US12056275B2

公开(公告)日：2024-08-06

申请号：US18048411

申请日：2022-10-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Daniel Miller ,  Renzo De Nardi ,  Richard Andrew Newcombe

IPC: G06F3/01 ,  G02B27/00 ,  G02B27/01 ,  G06N20/00 ,  G06V10/25 ,  G06V20/20 ,  G06V40/18

CPC classification number: G06F3/013 ,  G02B27/0093 ,  G02B27/0101 ,  G02B27/017 ,  G06N20/00 ,  G06V10/25 ,  G06V20/20 ,  G06V40/18 ,  G02B2027/014 ,  G02B2027/0178

Abstract: An Artificial Reality (AR) device captures an image of a physical environment surrounding a user wearing the AR device. A three-dimensional map corresponding to the physical environment is accessed. Then, a pose of the AR device relative to the three-dimensional map is determined based on first features of physical objects captured in the image and second features of object representations in the three-dimensional map. The device determines, using an eye tracker, a gaze of an eye of the user. Based on the gaze and the pose, a region of interest is computed in the three-dimensional map. Representations of physical devices in the region of interest of the three-dimensional map are identified. The device determines an intent of the user to interact with a physical device. A command is issued to the physical device based on the determined intent.

公开(公告)号：US20240411364A1

公开(公告)日：2024-12-12

申请号：US18789518

申请日：2024-07-30

Applicant: Meta Platforms Technologies, LLC

Inventor： Daniel Miller ,  Renzo De Nardi ,  Richard Andrew Newcombe

IPC: G06F3/01 ,  G02B27/00 ,  G02B27/01 ,  G06N20/00 ,  G06V10/25 ,  G06V20/20 ,  G06V40/18

Abstract: A non-transitory, computer-readable storage medium including executable instructions that, when executed by one or more processors, allow a user to interact with physical devices via an artificial-reality (AR) device is described. In response to a command received from the user of a head-wearable device, the instructions cause the one or more processors to (i) obtain an image of a physical environment surrounding the user wearing the head-wearable device, (ii) determine a region of interest, including one or more available physical devices, in the image based at least on a gaze of an eye of the user, (iii) determine, by a machine-learning model, an intent of the user to interact with a particular physical device of the one or more available physical devices, and (iv), based on the intent of the user to interact with a particular physical device, send a representation of the command to the particular physical device.

公开(公告)号：US20230130770A1

公开(公告)日：2023-04-27

申请号：US18048411

申请日：2022-10-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Daniel Miller ,  Renzo De Nardi ,  Richard Andrew Newcombe

IPC: G06F3/01 ,  G02B27/00 ,  G02B27/01 ,  G06V40/18 ,  G06V10/25 ,  G06V20/20 ,  G06N20/00

Abstract: An Artificial Reality (AR) device captures an image of a physical environment surrounding a user wearing the AR device. A three-dimensional map corresponding to the physical environment is accessed. Then, a pose of the AR device relative to the three-dimensional map is determined based on first features of physical objects captured in the image and second features of object representations in the three-dimensional map. The device determines, using an eye tracker, a gaze of an eye of the user. Based on the gaze and the pose, a region of interest is computed in the three-dimensional map. Representations of physical devices in the region of interest of the three-dimensional map are identified. The device determines an intent of the user to interact with a physical device. A command is issued to the physical device based on the determined intent.

公开(公告)号：US11816886B1

公开(公告)日：2023-11-14

申请号：US16021900

申请日：2018-06-28

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Richard Andrew Newcombe ,  Renzo De Nardi

IPC: G06K9/00 ,  G06V20/20 ,  G01S3/803 ,  G06V10/70 ,  G06V40/10 ,  H04R1/10 ,  G06T7/70 ,  G06F3/16

CPC classification number: G06V20/20 ,  G01S3/8034 ,  G06V10/768 ,  G06V40/10 ,  G06F3/16 ,  G06T7/70 ,  H04R1/1008

Abstract: A system may include a wearable apparatus dimensioned to be worn by a user about an axial region of the user’s body such that, when the wearable apparatus is worn by the user, the user’s field of view into a local environment is substantially free of a view of the wearable apparatus. The system may also include a machine-perception subsystem that is coupled to the wearable apparatus and that gathers information about the local environment by observing the local environment. Additionally, the system may include an experience-analysis subsystem that infers, based on the information about the local environment and information about the user, contextual information about an experience of the user in the local environment. Furthermore, the system may include a non-visual communication subsystem that outputs the contextual information about the experience of the user. Various other apparatuses, systems, and methods are also disclosed.

公开(公告)号：US11619814B1

公开(公告)日：2023-04-04

申请号：US15997259

申请日：2018-06-04

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： Richard Andrew Newcombe ,  Renzo De Nardi

IPC: G02B27/01 ,  G06T19/00 ,  G06T7/521 ,  G06T7/70 ,  G06F3/01

Abstract: A head-mounted display system may include a wearable frame securable to a user's head. The head-mounted display system may also include a mapping subsystem that maps a local environment of the user when the wearable frame is secured to the user's head. Additionally, the head-mounted display system may include a varifocal display apparatus mounted to the wearable frame and configured to direct computer-generated images toward the user's eyes at a variable focal length. Various other apparatuses, systems, and methods are also disclosed.