公开(公告)号：US20240346779A1

公开(公告)日：2024-10-17

申请号：US18630767

申请日：2024-04-09

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: G06T19/00 ,  G06T5/80 ,  H04N13/344

CPC classification number: G06T19/006 ,  G06T5/80 ,  H04N13/344

Abstract: A method includes determining that an inter-pupillary distance (IPD) between display lenses of a video see-through (VST) extended reality (XR) device has been adjusted with respect to a default IPD. The method also includes obtaining an image captured using a see-through camera of the VST XR device. The see-through camera is configured to capture images of a three-dimensional (3D) scene. The method further includes transforming the image to match a viewpoint of a corresponding one of the display lenses according to a change in IPD with respect to the default IPD in order to generate a transformed image. The method also includes correcting distortions in the transformed image based on one or more lens distortion coefficients corresponding to the change in IPD in order to generate a corrected image. In addition, the method includes initiating presentation of the corrected image on a display panel of the VST XR device.

公开(公告)号：US20240223742A1

公开(公告)日：2024-07-04

申请号：US18526726

申请日：2023-12-01

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: H04N13/344 ,  G06T19/00 ,  H04N13/128 ,  H04N13/239

CPC classification number: H04N13/344 ,  G06T19/006 ,  H04N13/128 ,  H04N13/239

Abstract: A method includes obtaining images of a scene captured using a stereo pair of imaging sensors of an XR device and depth data associated with the images, where the scene includes multiple objects. The method also includes obtaining volume-based 3D models of the objects. The method further includes, for one or more first objects, performing depth-based reprojection of the one or more 3D models of the one or more first objects to left and right virtual views based on one or more depths of the one or more first objects. The method also includes, for one or more second objects, performing constant-depth reprojection of the one or more 3D models of the one or more second objects to the left and right virtual views based on a specified depth. In addition, the method includes rendering the left and right virtual views for presentation by the XR device.

公开(公告)号：US20240223739A1

公开(公告)日：2024-07-04

申请号：US18360677

申请日：2023-07-27

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: H04N13/128 ,  G06T19/00

CPC classification number: H04N13/128 ,  G06T19/006 ,  H04N2013/0092

Abstract: A method includes obtaining first and second image frames of a scene. The method also includes providing the first image frame as input to an object segmentation model, where the object segmentation model is trained to generate first object segmentation predictions for objects in the scene and a depth or disparity map based on the first image frame. The method further includes generating second object segmentation predictions for the objects in the scene based on the second image frame. The method also includes determining boundaries of the objects in the scene based on the first and second object segmentation predictions. In addition, the method includes generating a virtual view for presentation on a display of an extended reality (XR) device based on the boundaries of the objects in the scene.

公开(公告)号：US20240129448A1

公开(公告)日：2024-04-18

申请号：US18353581

申请日：2023-07-17

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: H04N13/261 ,  G06T3/00 ,  G06T5/00 ,  G06T7/593 ,  G06T7/70 ,  G06T15/04 ,  H04N13/271

CPC classification number: H04N13/261 ,  G06T3/0093 ,  G06T5/005 ,  G06T7/593 ,  G06T7/70 ,  G06T15/04 ,  H04N13/271 ,  G06T2207/20081 ,  G06T2207/20212 ,  G06T2207/30244

Abstract: A method includes obtaining a 2D image captured using an imaging sensor. The 2D image is associated with an imaging sensor pose. The method also includes providing the 2D image, the imaging sensor pose, and one or more additional imaging sensor poses to at least one machine learning model that is trained to generate a texture map and a depth map for the imaging sensor pose and each additional imaging sensor pose. The method further includes generating a stereo image pair based on the texture maps and the depth maps. The stereo image pair represents a 2.5D view of the 2D image. The 2.5D view includes a pair of images each including multiple collections of pixels and, for each collection of pixels, a common depth associated with the pixels in the collection of pixels. In addition, the method includes initiating display of the stereo image pair on an XR device.

公开(公告)号：US11670063B2

公开(公告)日：2023-06-06

申请号：US17463037

申请日：2021-08-31

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: G06T19/20 ,  G06T5/00

CPC classification number: G06T19/20 ,  G06T5/005 ,  G06T2207/10024 ,  G06T2207/20221 ,  G06T2219/2012

Abstract: An electronic device that reprojects two-dimensional (2D) images to three-dimensional (3D) images includes a memory configured to store instructions, and a processor configured to execute the instructions to: propagate an intensity for at least one pixel of an image based on a depth guide of neighboring pixels of the at least one pixel, wherein the at least one pixel is considered a hole during 2D to 3D image reprojection; propagate a color for the at least one pixel based on an intensity guide of the neighboring pixels of the at least one pixel; and compute at least one weight for the at least one pixel based on the intensity and color propagation.

公开(公告)号：US20220292631A1

公开(公告)日：2022-09-15

申请号：US17402005

申请日：2021-08-13

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: G06T3/00 ,  G06T7/50 ,  G06F3/01 ,  G06T7/13

Abstract: Updating an image during real-time rendering of images by a display device can include determining a depth for each pixel of a color frame received from a source device and corresponding to the image. Each pixel's depth is determined by image-guided propagation of depths of sparse points extracted from a depth map generated at the source device. With respect to pixels corresponding to an extracted sparse depth point, image-guided depth propagation can include retaining the depth of the corresponding sparse depth point unchanged from the source depth map. With respect to each pixel corresponding to a non-sparse depth point, image-guided depth propagation can include propagating to the corresponding non-sparse depth point a depth of a sparse depth point lying within a neighborhood of the non-sparse depth point. Pixel coordinates of the color frame can be transformed for generating a space-warped rendering of the image.

公开(公告)号：US20210311307A1

公开(公告)日：2021-10-07

申请号：US16990779

申请日：2020-08-11

Applicant: Samsung Electronics Co., Ltd.

Inventor： Christopher A. Peri ,  Yingen Xiong ,  Lu Luo

IPC: G02B27/01 ,  G06T11/00 ,  G06T19/20 ,  G06F3/01

Abstract: A method includes obtaining, from a memory of an electronic device connected to a head mounted display (HMD), a first reference frame, wherein the first reference frame comprises a first set of pixels associated with a first time. The method includes, rendering, at the electronic device, a source image as a new frame, wherein the new frame includes a second set of pixels associated with a display to be provided by the HMD at a second time, and generating, by the electronic device, a differential frame, wherein the differential frame is based on a difference operation between pixels of the new frame with pixels of the first reference frame to identify pixels unique to the new frame. Still further, the method includes sending the differential frame to the HMD, and storing the new frame in the memory of the electronic device as a second reference frame.

公开(公告)号：US10691767B2

公开(公告)日：2020-06-23

申请号：US16362375

申请日：2019-03-22

Applicant: Samsung Electronics Co., Ltd.

Inventor： Christopher A. Peri

IPC: H04W24/00 ,  G06F16/955 ,  G06F16/9537 ,  H04L29/06 ,  G06K7/14 ,  G06T19/00

Abstract: An apparatus includes at least one image sensor configured to capture a plurality of images, at least one communication interface, at least one display, and at least one processor coupled to the at least one image sensor, the at least one communication interface, and the at least one display. The at least one processor is configured to identify a network address encoded into one or more first images of the plurality of captured images and transmit a first request to the network address. The at least one processor is also configured to identify a unique identifier encoded into one or more second images of the plurality of captured images and transmit a second request containing the unique identifier. The at least one processor is further configured to receive session information and output, to the at least one display, extended reality-related content obtained during a session associated with the session information.

公开(公告)号：US20200142942A1

公开(公告)日：2020-05-07

申请号：US16362375

申请日：2019-03-22

Applicant: Samsung Electronics Co., Ltd.

Inventor： Christopher A. Peri

IPC: G06F16/955 ,  G06F16/9537 ,  G06T19/00 ,  G06K7/14 ,  H04L29/06

Abstract: An apparatus includes at least one image sensor configured to capture a plurality of images, at least one communication interface, at least one display, and at least one processor coupled to the at least one image sensor, the at least one communication interface, and the at least one display. The at least one processor is configured to identify a network address encoded into one or more first images of the plurality of captured images and transmit a first request to the network address. The at least one processor is also configured to identify a unique identifier encoded into one or more second images of the plurality of captured images and transmit a second request containing the unique identifier. The at least one processor is further configured to receive session information and output, to the at least one display, extended reality-related content obtained during a session associated with the session information.

公开(公告)号：US10304207B2

公开(公告)日：2019-05-28

申请号：US15805750

申请日：2017-11-07

Applicant: Samsung Electronics Co., Ltd.

Inventor： Christopher A. Peri

IPC: G06T7/00 ,  G06T7/73 ,  G06K9/00 ,  G06T7/246 ,  G06T7/11

Abstract: A method, electronic device, and non-transitory computer readable medium for optical tracking are provided. The method includes identifying at least one object within an environment, from received image data. The method also includes assigning a volatility rating to the identified object. Additionally, the method includes determining one or more tracking points associated with the identified object. The method also includes determining a priority rating for the each of the one or more tracking points associated with the identified object based on the assigned volatility rating of the identified object. The method also includes selecting the one or more tracking points for optical tracking based on the priority rating.