公开(公告)号：US12033270B2

公开(公告)日：2024-07-09

申请号：US17972269

申请日：2022-10-24

Applicant: Meta Platforms Technologies, LLC

Inventor： Alessia Marra ,  Gioacchino Noris ,  Panya Inversin

IPC: H04N13/344 ,  G06F3/01 ,  G06F3/0346 ,  G06T7/70 ,  G06T15/20 ,  G06T19/00 ,  H04N5/272 ,  G02B27/01 ,  G06F3/04815 ,  G06V10/25

CPC classification number: G06T15/20 ,  G06F3/012 ,  G06F3/013 ,  G06F3/0346 ,  G06T7/70 ,  H04N13/344 ,  G02B27/017 ,  G06F3/011 ,  G06F3/04815 ,  G06F2200/00 ,  G06T19/006 ,  G06T2207/30244 ,  G06V10/25 ,  G09G2340/10 ,  G09G2340/12 ,  H04N5/272

Abstract: A method includes detecting an object of interest in a real environment and depth information of the object; determining one or more anchor locations in a three-dimensional space that correspond to a position of the object in the three-dimensional space; and generating a virtual surface anchored in the three-dimensional space. The method may further determine a pose of a camera when an image is captured and determine a region in the image that corresponds to the virtual surface. The method may further determine a first viewpoint of a first eye of the user; render a first output image based on (1) the first viewpoint relative to the virtual surface and (2) the image region corresponding to the virtual surface; and display the first output image on a first display of the computing device, the first display being configured to be viewed by the first eye of the user.

公开(公告)号：US11902288B2

公开(公告)日：2024-02-13

申请号：US18166512

申请日：2023-02-09

Applicant: Meta Platforms Technologies, LLC

Inventor： Michael James Lebeau ,  Manuel Ricardo Freire Santos ,  Aleksejs Anpilogovs ,  Alexander Sorkine Hornung ,  Björn Wanbo ,  Connor Treacy ,  Fangwei Lee ,  Federico Ruiz ,  Jonathan Mallinson ,  Jonathan Richard Mayoh ,  Marcus Tanner ,  Panya Inversin ,  Sarthak Ray ,  Sheng Shen ,  William Arthur Hugh Steptoe ,  Alessia Marra ,  Gioacchino Noris ,  Derrick Readinger ,  Jeffrey Wai-King Lock ,  Jeffrey Witthuhn ,  Jennifer Lynn Spurlock ,  Larissa Heike Laich ,  Javier Alejandro Sierra Santos

IPC: G06T19/00 ,  H04L9/40 ,  G06F3/02 ,  G06F3/01 ,  G06F3/14 ,  G06V40/10 ,  H04L65/401 ,  H04L65/403 ,  H04L65/70

CPC classification number: H04L63/107 ,  G06F3/011 ,  G06F3/0219 ,  G06F3/1423 ,  G06T19/003 ,  G06T19/006 ,  G06V40/107 ,  H04L65/4015 ,  H04L65/4046 ,  H04L65/70 ,  G06T2219/024

Abstract: Aspects of the present disclosure are directed to creating and administering artificial reality collaborative working environments and providing interaction modes for them. An XR work system can provide and control such artificial reality collaborative working environments to enable, for example, A) links between real-world surfaces and XR surfaces; B) links between multiple real-world areas to XR areas with dedicated functionality; C) maintaining access, while inside the artificial reality working environment, to real-world work tools such as the user's computer screen and keyboard; D) various hand and controller modes for different interaction and collaboration modalities; E) use-based, multi-desk collaborative room configurations; and F) context-based auto population of users and content items into the artificial reality working environment.

公开(公告)号：US20230125961A1

公开(公告)日：2023-04-27

申请号：US18145603

申请日：2022-12-22

Applicant: Meta Platforms Technologies, LLC

Inventor： Alexander Sorkine Hornung ,  Panya Inversin

IPC: G06T15/20 ,  H04N13/239

Abstract: A method includes receiving video data of a user, the video data comprising a first captured image and a second captured image, generating a two-dimensional planar proxy of the user, determining a pose comprising a location and orientation of the two-dimensional planar proxy within a three-dimensional virtual environment, rendering one or more display images for one or more displays of an artificial-reality device based on the two-dimensional planar proxy having the determined pose and at least one of the first and second captured images, displaying the rendered one or more display images using the one or more displays, respectively, determining that a viewing angle of the artificial-reality device relative to the two-dimensional planar proxy exceeds a predetermined maximum threshold, and based on the determination that the viewing angle exceeds the predetermined maximum threshold, ceasing to display the one or more display images.

公开(公告)号：US20230037750A1

公开(公告)日：2023-02-09

申请号：US17972269

申请日：2022-10-24

Applicant: Meta Platforms Technologies, LLC

Inventor： Alessia Marra ,  Gioacchino Noris ,  Panya Inversin

IPC: G06T15/20 ,  G06T7/70 ,  H04N13/344 ,  G06F3/01 ,  G06F3/0346

Abstract: A method includes detecting an object of interest in a real environment and depth information of the object; determining one or more anchor locations in a three-dimensional space that correspond to a position of the object in the three-dimensional space; and generating a virtual surface anchored in the three-dimensional space. The method may further determine a pose of a camera when an image is captured and determine a region in the image that corresponds to the virtual surface. The method may further determine a first viewpoint of a first eye of the user; render a first output image based on (1) the first viewpoint relative to the virtual surface and (2) the image region corresponding to the virtual surface; and display the first output image on a first display of the computing device, the first display being configured to be viewed by the first eye of the user.

公开(公告)号：US11481960B2

公开(公告)日：2022-10-25

申请号：US17138414

申请日：2020-12-30

Applicant: Meta Platforms Technologies, LLC

Inventor： Alessia Marra ,  Gioacchino Noris ,  Panya Inversin

IPC: G06F3/01 ,  G06T7/70 ,  H04N13/344 ,  G06F3/0346 ,  G06V10/25 ,  H04N5/272 ,  G06T19/00 ,  G02B27/01 ,  G06T15/20 ,  G06F3/04815

Abstract: A method includes a computing system tracking motions performed by a hand of a user, determining one or more anchor locations in a three-dimensional space, and generating a virtual surface anchored in the three-dimensional space. An image of a real environment is captured using a camera worn by the user, and a pose of the camera when the image is captured is determined. The computing system determines a first viewpoint of a first eye of the user and a region in the image that, as viewed from the camera, corresponds to the virtual surface. The computing system renders an output image based on (1) the first viewpoint relative to the virtual surface and (2) the image region corresponding to the virtual surface, and displays the output image on a first display of the device, the first display being configured to be viewed by the first eye of the user.

公开(公告)号：US12105866B2

公开(公告)日：2024-10-01

申请号：US17673682

申请日：2022-02-16

Applicant: META PLATFORMS TECHNOLOGIES, LLC

Inventor： David Frederick Geisert ,  Alessia Marra ,  Gioacchino Noris ,  Panya Inversin

IPC: G06F3/01 ,  G02B27/01 ,  G06T19/00

CPC classification number: G06F3/011 ,  G02B27/0101 ,  G06T19/006 ,  G02B2027/014

Abstract: In one embodiment, a method includes capturing, by a first VR display device, one or more frames of a shared real-world environment. The VR display device identifies one or more anchor points within the shared real-world environment from the one or more frames. The first VR display device receives localization information with respect to a second VR display device in the shared real-world environment and determines a pose of the first VR display device with respect to the second VR display device based on the localization information. A first output image is rendered for one or more displays of the first VR display device. The rendered image may comprise a proximity warning with respect to the second VR display device based on determining the pose of the first VR display device with respect to the second VR display device is within a threshold distance.

公开(公告)号：US20230259194A1

公开(公告)日：2023-08-17

申请号：US17673682

申请日：2022-02-16

Applicant: Meta Platforms Technologies, LLC

Inventor： David Frederick Geisert ,  Alessia Marra ,  Gioacchino Noris ,  Panya Inversin

IPC: G06F3/01 ,  G06T19/00 ,  G02B27/01

CPC classification number: G06F3/011 ,  G06T19/006 ,  G02B27/0101 ,  G02B2027/014

Abstract: In one embodiment, a method includes capturing, by a first VR display device, one or more frames of a shared real-world environment. The VR display device identifies one or more anchor points within the shared real-world environment from the one or more frames. The first VR display device receives localization information with respect to a second VR display device in the shared real-world environment and determines a pose of the first VR display device with respect to the second VR display device based on the localization information. A first output image is rendered for one or more displays of the first VR display device. The rendered image may comprise a proximity warning with respect to the second VR display device based on determining the pose of the first VR display device with respect to the second VR display device is within a threshold distance.

公开(公告)号：US20230131667A1

公开(公告)日：2023-04-27

申请号：US18069164

申请日：2022-12-20

Applicant: Meta Platforms Technologies, LLC

Inventor： Adrian Brian Ratter ,  Alessia Marra ,  Yugeng He ,  Panya Inversin

IPC: G06F3/04815 ,  G06T7/70 ,  G02B27/00 ,  G02B27/01 ,  G06T7/10

Abstract: A method includes accessing an image of a physical environment of a user, the image depicting a physical input device and a physical hand of the user, determining that a contrast between the physical input device and the physical hand depicted in the image is lower than a predetermined threshold, modifying the image to increase the contrast, determining a pose of the physical input device, generating a three-dimensional model representing the physical hand of the user, generating an image mask by projecting the three-dimensional model onto an image plane, generating, a cropped image depicting at least the physical hand of the user in the image, rendering, based on the perspective of the user and the pose of the physical input device, a virtual input device to represent the physical input device, and displaying the cropped image depicting at least the physical hand over the rendered virtual input device.

公开(公告)号：US11582245B2

公开(公告)日：2023-02-14

申请号：US17085652

申请日：2020-10-30

Applicant: Meta Platforms Technologies, LLC

Inventor： Michael James LeBeau ,  Manuel Ricardo Freire Santos ,  Aleksejs Anpilogovs ,  Alexander Sorkine Hornung ,  Bjorn Wanbo ,  Connor Treacy ,  Fangwei Lee ,  Federico Ruiz ,  Jonathan Mallinson ,  Jonathan Richard Mayoh ,  Marcus Tanner ,  Panya Inversin ,  Sarthak Ray ,  Sheng Shen ,  William Arthur Hugh Steptoe ,  Alessia Marra ,  Gioacchino Noris ,  Derrick Readinger ,  Jeffrey Wai-King Lock ,  Jeffrey Witthuhn ,  Jennifer Lynn Spurlock ,  Larissa Heike Laich ,  Javier Alejandro Sierra Santos

IPC: H04L65/70 ,  H04L9/40 ,  G06F3/02 ,  G06F3/01 ,  G06F3/14 ,  G06V40/10 ,  G06T19/00 ,  H04L65/401 ,  H04L65/403

Abstract: Aspects of the present disclosure are directed to creating and administering artificial reality collaborative working environments and providing interaction modes for them. An XR work system can provide and control such artificial reality collaborative working environments to enable, for example, A) links between real-world surfaces and XR surfaces; B) links between multiple real-world areas to XR areas with dedicated functionality; C) maintaining access, while inside the artificial reality working environment, to real-world work tools such as the user's computer screen and keyboard; D) various hand and controller modes for different interaction and collaboration modalities; E) use-based, multi-desk collaborative room configurations; and F) context-based auto population of users and content items into the artificial reality working environment.

公开(公告)号：US11538214B2

公开(公告)日：2022-12-27

申请号：US17093228

申请日：2020-11-09

Applicant: Meta Platforms Technologies, LLC

Inventor： Alexander Sorkine Hornung ,  Panya Inversin

IPC: G06T15/00 ,  G06T15/20 ,  H04N13/239

Abstract: A method includes receiving, through a network by a computing system associated with an artificial-reality device, video data of a user of a second computing system comprising a first and second image of the user. The first and second images may be captured concurrently by a first camera and a second camera of the second computing system, respectively. The computing system generates a planar proxy for displaying the user and determines a pose for the planar proxy within a three-dimensional virtual environment. The computing system renders a left image for a left-eye display and a right image for a right-eye display of the artificial-reality device based on the planar proxy having the determined pose and the first image, and the planar proxy having the determined pose and the second image, respectively. The computing system displays the rendered left image and right image using the left-eye display and right-eye display, respectively.