公开(公告)号：US12198247B2

公开(公告)日：2025-01-14

申请号：US17806029

申请日：2022-06-08

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher Peri

IPC: G06T15/00 ,  G06T7/50 ,  H04N13/128 ,  H04N13/225

Abstract: A method includes receiving, from a camera, one or more frames of image data of a scene comprising a background and one or more three-dimensional objects, wherein each frame comprises a raster of pixels of image data; detecting layer information of the scene, wherein the layer information is associated with a depth-based distribution of the pixels in the one or more frames; and determining a multi-layer model for the scene, the multi-layer model comprising a plurality of discrete layers comprising first and second discrete layers, wherein each discrete layer is associated with a unique depth value relative to the camera. The method further includes mapping the pixels to the layers of the plurality of discrete layers; rendering the pixels as a first image of the scene as viewed from a first perspective; and rendering the pixels as a second image of the scene as viewed from a second perspective.

公开(公告)号：US20240233098A1

公开(公告)日：2024-07-11

申请号：US18360683

申请日：2023-07-27

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: G06T5/00 ,  G06T3/00 ,  G06T19/00 ,  G06V10/56

CPC classification number: G06T5/006 ,  G06T3/0093 ,  G06T19/006 ,  G06V10/56 ,  G06T2207/10024

Abstract: A method includes obtaining an image captured by a see-through camera of a video see-through (VST) augmented reality (AR) device. The method also includes identifying a first distortion created by at least one see-through camera lens and identifying a second distortion created by at least one display lens of the VST AR device. The method further includes determining a combined distortion based on the first distortion and the second distortion. The method also includes pre-warping the image to offset the combined distortion such that the image is not distorted when the pre-warped image is viewed by a user through the at least one display lens of the VST AR device. In addition, the method includes presenting the pre-warped image to the user on at least one display of the VST AR device.

公开(公告)号：US20240078765A1

公开(公告)日：2024-03-07

申请号：US18353610

申请日：2023-07-17

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong

IPC: G06T19/00 ,  G06F3/14 ,  G06T5/00 ,  G06V10/25

CPC classification number: G06T19/006 ,  G06F3/14 ,  G06T5/005 ,  G06V10/25

Abstract: A method includes generating a virtual view image and a virtual depth map based on an image captured using a see-through camera and a corresponding depth map. The virtual view image and the virtual depth map include holes for which image data or depth data cannot be determined. The method also includes searching one or more previous images to locate a region in at least one previous image that includes missing pixels in the holes. The method further includes at least partially filling the holes in the virtual view image and the virtual depth map with image data and depth data associated with the located region to generate a filled virtual view image and a filled virtual depth map. In addition, the method includes generating a virtual view to present on a display panel of a VST AR device using the filled virtual view image and the filled virtual depth map.

公开(公告)号：US11741671B2

公开(公告)日：2023-08-29

申请号：US17500817

申请日：2021-10-13

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: G06T17/20 ,  G06N20/00 ,  G06T3/40 ,  G06T7/55 ,  G06T19/00 ,  G06T19/20

CPC classification number: G06T17/20 ,  G06N20/00 ,  G06T3/4053 ,  G06T7/55 ,  G06T19/006 ,  G06T19/20 ,  G06T2207/10024

Abstract: Generating a 3D scene reconstruction using depth fusion can include creating a high-resolution sparse depth map by mapping sensor depths from a low-resolution depth map to points corresponding to pixels of a high-resolution color image of a scene. The high-resolution sparse depth map can have the same resolution as the high-resolution color image. A fused sparse depth map can be produced by combining the high-resolution sparse depth map with sparse depths reconstructed from the high-resolution color image. The high-resolution dense depth map can be generated based on fused sparse depths of the fused sparse depth map.

公开(公告)号：US20230245322A1

公开(公告)日：2023-08-03

申请号：US17875429

申请日：2022-07-28

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher Peri

IPC: G06T7/20 ,  G06T7/593 ,  G06T17/00 ,  H04N13/106

CPC classification number: G06T7/20 ,  G06T7/593 ,  G06T17/00 ,  H04N13/106 ,  G06T2207/10012 ,  G06T2207/10021 ,  G06T2207/30244 ,  H04N2013/0081

Abstract: In one embodiment, a method includes identifying, in each image of a stereoscopic pair of images of a scene at a particular time, every pixel as either a static pixel corresponding to a portion of a scene that does not have local motion at that time or a dynamic pixel corresponding to a portion of a scene that has local motion at that time. For each static pixel, the method includes comparing each of a plurality of depth calculations for the pixel, and when the depth calculations differ by at least a threshold amount, then re-labeling that pixel as a dynamic pixel. For each dynamic pixel, the method includes comparing a geometric 3D calculation for the pixel with a temporal 3D calculation for that pixel, and when the geometric 3D calculation and the temporal 3D calculation are within a threshold amount, then re-labeling the pixel as a static pixel.

公开(公告)号：US20230137199A1

公开(公告)日：2023-05-04

申请号：US17696729

申请日：2022-03-16

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: G09G3/20

Abstract: A system and method for display distortion calibration are configured to capture distortion with image patterns and calibrate distortion with ray tracing for an optical pipeline with lenses. The system includes an image sensor and a processor to perform the method for display distortion calibration. The method includes generating an image pattern to encode display image pixels by encoding display distortion associated with a plurality of image patterns. The method also includes determining a distortion of the image pattern resulting from a lens on a head-mounted display (HMD) and decoding the distorted image patterns to obtain distortion of pixels on a display. A lookup table is created of angular distortion of all the pixels on the display. The method further includes providing a compensation factor for the distortion by creating distortion correction based on the lookup table of angular distortion.

公开(公告)号：US11961184B2

公开(公告)日：2024-04-16

申请号：US17696746

申请日：2022-03-16

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: G06T17/20 ,  G06T7/12 ,  G06T19/00

CPC classification number: G06T17/20 ,  G06T7/12 ,  G06T19/006 ,  G06T2200/04 ,  G06T2200/08 ,  G06T2207/10012 ,  G06T2207/20021

Abstract: A system and method for 3D reconstruction with plane and surface reconstruction, scene parsing, depth reconstruction with depth fusion from different sources. The system includes display and a processor to perform the method for 3D reconstruction with plane and surface reconstruction. The method includes dividing a scene of an image frame into one or more plane regions and one or more surface regions. The method also includes generating reconstructed planes by performing plane reconstruction based on the one or more plane regions. The method also includes generating reconstructed surfaces by performing surface reconstruction based on the one or more surface regions. The method further includes creating the 3D scene reconstruction by integrating the reconstructed planes and the reconstructed surfaces.

公开(公告)号：US20240046583A1

公开(公告)日：2024-02-08

申请号：US18353579

申请日：2023-07-17

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: G06T19/00 ,  G06T7/73 ,  G06T7/593

CPC classification number: G06T19/006 ,  G06T7/73 ,  G06T7/593 ,  G06T2207/30244 ,  G06T2207/10021 ,  G06T2207/20224

Abstract: A method includes obtaining images of a scene and corresponding position data of a device that captures the images. The method also includes determining position data and direction data associated with camera rays passing through keyframes of the images. The method further includes using a position-dependent multilayer perceptron (MLP) and a direction-dependent MLP to create sparse feature vectors. The method also includes storing the sparse feature vectors in at least one data structure. The method further includes receiving a request to render the scene on an augmented reality (AR) device associated with a viewing direction. In addition, the method includes rendering the scene associated with the viewing direction using the sparse feature vectors in the at least one data structure.

公开(公告)号：US20240046576A1

公开(公告)日：2024-02-08

申请号：US18136211

申请日：2023-04-18

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong

IPC: G06T19/00 ,  H04N13/327 ,  H04N13/344 ,  G06T7/80 ,  H04N13/246 ,  H04N13/257 ,  G02B27/01

CPC classification number: G06T19/006 ,  H04N13/327 ,  H04N13/344 ,  G06T7/80 ,  H04N13/246 ,  H04N13/257 ,  G02B27/0172 ,  G06T2207/10021 ,  G06T2207/30244 ,  G06T2207/10024 ,  G02B2027/0138

Abstract: In one embodiment, a method includes capturing, by a calibration camera, a calibration pattern displayed on a display of a video see-through AR system, where the calibration camera is located at an eye position for viewing content on the video see-through AR system. The method further includes determining, based on the captured calibration pattern, one or more system parameters of the video see-through AR system that represent combined distortion caused by a display lens of the video see-through AR system and caused by a camera lens of the see-through camera; determining, based on the one or more system parameters and on one or more camera parameters that represent distortion caused by the see-through camera, one or more display-lens parameters that represent distortion caused by the display lens; and storing the one or more system parameters and the one more display-lens parameters as a calibration for the system.

公开(公告)号：US20230140170A1

公开(公告)日：2023-05-04

申请号：US17811028

申请日：2022-07-06

Applicant: Samsung Electronics Co., Ltd.

Inventor： Yingen Xiong ,  Christopher A. Peri

IPC: G06T19/00 ,  G06T7/593 ,  G06T7/73 ,  G06F3/01 ,  H04N13/239

Abstract: A method includes obtaining first and second image data of a real-world scene, performing feature extraction to obtain first and second feature maps, and performing pose tracking based on at least one of the first image data, second image data, and pose data to obtain a 6DOF pose of an apparatus. The method also includes generating, based on the 6DOF pose, first feature map, and second feature map, a disparity map between the image data and generating an initial depth map based on the disparity map. The method further includes generating a dense depth map based on the initial depth map and a camera model and generating, based on the dense depth map, a three-dimensional reconstruction of at least pail of the scene. In addition, the method includes rendering an AR or XR display that includes one or more virtual objects positioned to contact one or more surfaces of the reconstruction.