公开(公告)号：US12033270B2

公开(公告)日：2024-07-09

申请号：US17972269

申请日：2022-10-24

Applicant: Meta Platforms Technologies, LLC

Inventor： Alessia Marra ,  Gioacchino Noris ,  Panya Inversin

IPC: H04N13/344 ,  G06F3/01 ,  G06F3/0346 ,  G06T7/70 ,  G06T15/20 ,  G06T19/00 ,  H04N5/272 ,  G02B27/01 ,  G06F3/04815 ,  G06V10/25

CPC classification number: G06T15/20 ,  G06F3/012 ,  G06F3/013 ,  G06F3/0346 ,  G06T7/70 ,  H04N13/344 ,  G02B27/017 ,  G06F3/011 ,  G06F3/04815 ,  G06F2200/00 ,  G06T19/006 ,  G06T2207/30244 ,  G06V10/25 ,  G09G2340/10 ,  G09G2340/12 ,  H04N5/272

Abstract: A method includes detecting an object of interest in a real environment and depth information of the object; determining one or more anchor locations in a three-dimensional space that correspond to a position of the object in the three-dimensional space; and generating a virtual surface anchored in the three-dimensional space. The method may further determine a pose of a camera when an image is captured and determine a region in the image that corresponds to the virtual surface. The method may further determine a first viewpoint of a first eye of the user; render a first output image based on (1) the first viewpoint relative to the virtual surface and (2) the image region corresponding to the virtual surface; and display the first output image on a first display of the computing device, the first display being configured to be viewed by the first eye of the user.

公开(公告)号：US11902288B2

公开(公告)日：2024-02-13

申请号：US18166512

申请日：2023-02-09

Applicant: Meta Platforms Technologies, LLC

Inventor： Michael James Lebeau ,  Manuel Ricardo Freire Santos ,  Aleksejs Anpilogovs ,  Alexander Sorkine Hornung ,  Björn Wanbo ,  Connor Treacy ,  Fangwei Lee ,  Federico Ruiz ,  Jonathan Mallinson ,  Jonathan Richard Mayoh ,  Marcus Tanner ,  Panya Inversin ,  Sarthak Ray ,  Sheng Shen ,  William Arthur Hugh Steptoe ,  Alessia Marra ,  Gioacchino Noris ,  Derrick Readinger ,  Jeffrey Wai-King Lock ,  Jeffrey Witthuhn ,  Jennifer Lynn Spurlock ,  Larissa Heike Laich ,  Javier Alejandro Sierra Santos

IPC: G06T19/00 ,  H04L9/40 ,  G06F3/02 ,  G06F3/01 ,  G06F3/14 ,  G06V40/10 ,  H04L65/401 ,  H04L65/403 ,  H04L65/70

CPC classification number: H04L63/107 ,  G06F3/011 ,  G06F3/0219 ,  G06F3/1423 ,  G06T19/003 ,  G06T19/006 ,  G06V40/107 ,  H04L65/4015 ,  H04L65/4046 ,  H04L65/70 ,  G06T2219/024

Abstract: Aspects of the present disclosure are directed to creating and administering artificial reality collaborative working environments and providing interaction modes for them. An XR work system can provide and control such artificial reality collaborative working environments to enable, for example, A) links between real-world surfaces and XR surfaces; B) links between multiple real-world areas to XR areas with dedicated functionality; C) maintaining access, while inside the artificial reality working environment, to real-world work tools such as the user's computer screen and keyboard; D) various hand and controller modes for different interaction and collaboration modalities; E) use-based, multi-desk collaborative room configurations; and F) context-based auto population of users and content items into the artificial reality working environment.

公开(公告)号：US11861757B2

公开(公告)日：2024-01-02

申请号：US17930181

申请日：2022-09-07

Applicant: Meta Platforms Technologies, LLC

Inventor： James Allan Booth ,  Mahdi Salmani Rahimi ,  Gioacchino Noris

IPC: G06T19/00 ,  G06T15/20 ,  G06N20/00 ,  G06V40/10

CPC classification number: G06T19/006 ,  G06N20/00 ,  G06T15/205 ,  G06V40/10

Abstract: The disclosed artificial reality system can provide a user self representation in an artificial reality environment based on a self portion from an image of the user. The artificial reality system can generate the self representation by applying a machine learning model to classify the self portion of the image. The machine learning model can be trained to identify self portions in images based on a set of training images, with portions tagged as either depicting a user from a self-perspective or not. The artificial reality system can display the self portion as a self representation in the artificial reality environment by positioning them in the artificial reality environment relative to the user's perspective in the artificial reality environment. The artificial reality system can also identify movements of the user and can adjust the self representation to match the user's movement, providing more accurate self representations.

公开(公告)号：US20230177833A1

公开(公告)日：2023-06-08

申请号：US18159234

申请日：2023-01-25

Applicant: Meta Platforms Technologies, LLC

Inventor： Alexandru-Eugen Ichim ,  Sarthak Ray ,  Alexander Sorkine Hornung ,  Gioacchino Noris ,  Gaurav Chaurasia ,  Jan Oberländer

IPC: G06V20/20 ,  G06T7/73 ,  G06F3/01 ,  G06T15/20 ,  G06T19/00

CPC classification number: G06V20/20 ,  G06T7/74 ,  G06F3/013 ,  G06T15/205 ,  G06T19/006 ,  G06T2207/10028

Abstract: A system generates a plurality of spatial points based on depth measurements of physical objects. The system determines, based on the plurality of spatial points, an occupancy score for each voxel within a plurality of voxels. The system identifies, based on a gaze of the user, a first set of occupied voxels that are in a field of view of the user and a second set of occupied voxels that are outside the field of view of the user. The system updates the occupancy scores of the first set of occupied voxels by temporally decaying one or more of the plurality of spatial points within the first set of occupied voxels. The system maintains the occupancy scores of the second set of occupied voxels. The system detects intrusions in a predefined subspace within a physical space based on the updated occupancy scores of the first set of occupied voxels.

公开(公告)号：US11574474B2

公开(公告)日：2023-02-07

申请号：US17479961

申请日：2021-09-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Alexandru-Eugen Ichim ,  Sarthak Ray ,  Alexander Sorkine Hornung ,  Gioacchino Noris ,  Gaurav Chaurasia ,  Jan Oberländer

IPC: G06V20/20 ,  G06T7/73 ,  G06F3/01 ,  G06T15/20 ,  G06T19/00

Abstract: A system generates a plurality of spatial points based on depth measurements of physical objects. The system determines, based on the plurality of spatial points, an occupancy score for each voxel within a plurality of voxels. The system identifies, based on a gaze of the user, a first set of occupied voxels that are in a field of view of the user and a second set of occupied voxels that are outside the field of view of the user. The system updates the occupancy scores of the first set of occupied voxels by temporally decaying one or more of the plurality of spatial points within the first set of occupied voxels. The system maintains the occupancy scores of the second set of occupied voxels. The system detects intrusions in a predefined subspace within a physical space based on the updated occupancy scores of the first set of occupied voxels.

公开(公告)号：US12067673B2

公开(公告)日：2024-08-20

申请号：US17965500

申请日：2022-10-13

Applicant: Meta Platforms Technologies, LLC

Inventor： Alberto Garcia Garcia ,  Gioacchino Noris ,  Gian Diego Tipaldi

IPC: G06T15/00 ,  G06T7/11 ,  G06T15/20 ,  G06T15/40

CPC classification number: G06T15/40 ,  G06T7/11 ,  G06T15/20 ,  G06T2207/10028

Abstract: In one embodiment, a computing system determines one or more depth measurements associated with a first physical object. The system captures an image including image data associated with the first physical object. The system identifies and associates a plurality of first pixels with a first representative depth value based on the one or more depth measurements. The system determines, for an output pixel of an output image, that (1) a portion of a virtual object is visible from a viewpoint and (2) the output pixel overlaps with a portion of the first physical object. The system determines that the portion of the first physical object is associated with the plurality of first pixels and renders the output image from the viewpoint. Occlusion at the output pixel is determined based on a comparison between the first representative depth value and a depth value associated with the portion of the virtual object.

公开(公告)号：US12008717B2

公开(公告)日：2024-06-11

申请号：US17369267

申请日：2021-07-07

Applicant: Meta Platforms Technologies, LLC

Inventor： Gioacchino Noris ,  Michal Hlavac ,  Paul Timothy Furgale ,  Johannes Joachim Schmid ,  Anush Mohan ,  Christopher Richard Tanner

IPC: G06F16/909 ,  G06T19/00

CPC classification number: G06T19/00

Abstract: An artificial reality environment (XRE) schema is defined that supports anel-controlling interactions between various artificial reality actors. The XRE schema includes a set of definitions for an XRE, independent of type of artificial reality device. The definitions in the XRE schema can include standards for both interfaces and data objects. The XRE schema can define XR elements in terms of entities and components of a space, organized according to a hierarchy. Each entity can represent a real or virtual object or space, within the XRE, defined by a name and a collection of one or more components. Each component (as part of an entity) can define aspects and expose information about the entity. The XRE schema can specify structures that that allow actors (e.g., producers, instantiators, and consumers) to define and perform actions in relation to XRE elements.

公开(公告)号：US11830148B2

公开(公告)日：2023-11-28

申请号：US16944045

申请日：2020-07-30

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： James Allan Booth ,  Gaurav Chaurasia ,  Alexandru-Eugen Ichim ,  Alex Locher ,  Gioacchino Noris ,  Alexander Sorkine Hornung ,  Manuel Werlberger

IPC: G06T19/00 ,  G06T15/04 ,  G06T17/20

CPC classification number: G06T19/006 ,  G06T15/04 ,  G06T17/20

Abstract: A mixed reality (MR) simulation system includes a console and a head mounted device (HMD). The MR system captures stereoscopic images from a real-world environment using outward-facing stereoscopic cameras mounted to the HMD. The MR system preprocesses the stereoscopic images to maximize contrast and then extracts a set of features from those images, including edges or corners, among others. For each feature, the MR system generates one or more two-dimensional (2D) polylines. Then, the MR system triangulates between 2D polylines found in right side images and corresponding 2D polylines found in left side images to generate a set of 3D polylines. The MR system interpolates between 3D vertices included in the 3D polylines or extrapolates additional 3D vertices, thereby generating a geometric reconstruction of the real-world environment. The MR system may map textures derived from the real-world environment onto the geometric representation faster than the geometric reconstruction is updated.

公开(公告)号：US20230127307A1

公开(公告)日：2023-04-27

申请号：US17969622

申请日：2022-10-19

Applicant: Meta Platforms Technologies, LLC

Inventor： Gioacchino Noris ,  Sony Nguyen ,  Andrea Alejandra Cohen ,  Anush Mohan ,  Matthew Banks

IPC: G06T15/06 ,  G06T19/00 ,  G06T15/08 ,  G06T17/00

Abstract: In particular embodiments, a computing system may initiate a scene capture process to capture a scene. The scene may include one or more of planes or objects. The system may send a first set of instructions to outline one or more planes of the scene. The system may cast a first set of rays to outline the one or more planes. The system may create the one or more planes based on the first set of rays. The system may send a second set of instructions to outline one or more objects of the scene. The system may cast a second set of rays to outline the one or more objects. The system may create the one or more objects based on the second set of rays. The system may generate a scene model of the scene based on the one or more planes and the one or more objects.

公开(公告)号：US12277642B2

公开(公告)日：2025-04-15

申请号：US17969619

申请日：2022-10-19

Applicant: Meta Platforms Technologies, LLC

Inventor： Gioacchino Noris ,  Hongbing Hu ,  Anush Mohan

IPC: G06T15/06 ,  G06T15/08 ,  G06T17/00 ,  G06T19/00 ,  G06T19/20

Abstract: In particular embodiments, a computing system may initiate a scene alignment process to align a previous map of a scene with a current map of the scene. The system may send instructions to a user wearing an artificial-reality system to select a set of entities in the scene. The system may receive a selection of the set of entities in the scene. The system may determine a particular point in the scene based on an intersection of selected set of entities. The system may align the previous map with the current map based on the particular point in the scene. The system may load a scene model associated with the previous map into the current map.