公开(公告)号：US20240331181A1

公开(公告)日：2024-10-03

申请号：US18434684

申请日：2024-02-06

Applicant: Adeia Imaging LLC

Inventor： Florian Ciurea ,  Kartik Venkataraman ,  Gabriel Molina ,  Dan Lelescu

IPC: G06T7/593 ,  G02B27/00 ,  G06T7/557 ,  G06T7/80 ,  G06T15/20 ,  H04N13/00 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243 ,  H04N23/16

CPC classification number: G06T7/593 ,  G02B27/0075 ,  G06T7/557 ,  G06T7/85 ,  G06T15/20 ,  H04N13/128 ,  H04N13/232 ,  H04N13/243 ,  H04N23/16 ,  G06T2200/21 ,  G06T2207/10012 ,  G06T2207/10024 ,  G06T2207/10052 ,  H04N2013/0081 ,  H04N2013/0088

Abstract: Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

公开(公告)号：US11991342B2

公开(公告)日：2024-05-21

申请号：US17354517

申请日：2021-06-22

Applicant: Niantic, Inc.

Inventor： Clément Godard ,  Oisin MacAodha ,  Michael Firman ,  Gabriel J. Brostow

IPC: H04N13/271 ,  G06T7/579 ,  G06T7/593 ,  G06T7/73 ,  H04N13/00

CPC classification number: H04N13/271 ,  G06T7/579 ,  G06T7/593 ,  G06T7/73 ,  G06T2207/10016 ,  G06T2207/10028 ,  G06T2207/20081 ,  H04N2013/0081 ,  H04N2013/0088

Abstract: A method for training a depth estimation model and methods for use thereof are described. Images are acquired and input into a depth model to extract a depth map for each of the plurality of images based on parameters of the depth model. The method includes inputting the images into a pose decoder to extract a pose for each image. The method includes generating a plurality of synthetic frames based on the depth map and the pose for each image. The method includes calculating a loss value with an input scale occlusion and motion aware loss function based on a comparison of the synthetic frames and the images. The method includes adjusting the plurality of parameters of the depth model based on the loss value. The trained model can receive an image of a scene and generate a depth map of the scene according to the image.

公开(公告)号：US11876949B2

公开(公告)日：2024-01-16

申请号：US17714340

申请日：2022-04-06

Applicant: Avalon Holographics Inc.

Inventor： Matthew Hamilton ,  Chuck Rumbolt ,  Donovan Benoit ,  Matthew Troke ,  Robert Lockyer ,  Thomas Butyn

IPC: H04N13/161 ,  H04N19/119 ,  H04N19/146 ,  H04N19/187 ,  G06T7/557 ,  G06T15/06 ,  G06T17/00 ,  H04N13/00

CPC classification number: H04N13/161 ,  G06T7/557 ,  G06T15/06 ,  G06T17/00 ,  H04N19/119 ,  H04N19/146 ,  H04N19/187 ,  H04N2013/0088

Abstract: A system and methods for a CODEC driving a real-time light field display for multi-dimensional video streaming, interactive gaming and other light field display applications is provided applying a layered scene decomposition strategy. Multi-dimensional scene data including information on transparency of surfaces is divided into a plurality of data layers of increasing depths as the distance between a given layer and the display surface increases. Data layers are sampled using a plenoptic sampling scheme and rendered using hybrid rendering, such as perspective and oblique rendering, to encode light fields corresponding to each data layer. The resulting compressed, (layered) core representation of the multi-dimensional scene data is produced at predictable rates, reconstructed and merged at the light field display in real-time by applying view synthesis protocols, including edge adaptive interpolation, to reconstruct pixel arrays in stages (e.g. columns then rows) from reference elemental images.

公开(公告)号：US11694313B2

公开(公告)日：2023-07-04

申请号：US17018943

申请日：2020-09-11

Applicant: UNITY TECHNOLOGIES SF

Inventor： Kimball D. Thurston, III ,  Peter M. Hillman

IPC: G06T5/00 ,  G06T15/08 ,  H04N13/122 ,  G06T15/06 ,  G06V10/75 ,  G06V10/80 ,  G06V20/40 ,  H04N13/00

CPC classification number: G06T5/005 ,  G06T15/06 ,  G06T15/08 ,  G06V10/751 ,  G06V10/803 ,  G06V20/40 ,  H04N13/122 ,  H04N2013/0077 ,  H04N2013/0088

Abstract: An imagery processing system determines pixel color values for pixels of captured imagery from volumetric data, providing alternative pixel color values. A main imagery capture device, such as a camera, captures main imagery such as still images and/or video sequences, of a live action scene. Alternative devices capture imagery of the live action scene, in some spectra and form, and capture information related to pixel color values for multiple depths of a scene, which can be processed to provide reconstruction.

公开(公告)号：US11670022B2

公开(公告)日：2023-06-06

申请号：US16884139

申请日：2020-05-27

Applicant: SAMSUNG ELECTRONICS CO., LTD.

Inventor： Ji-Hun Park ,  Tai-Geun Kang ,  Ji-Hie Kim ,  Dug-In Lyu ,  Jong-Deok Choi

IPC: G06T11/60 ,  G06T19/00 ,  H04N13/156 ,  H04N13/106 ,  H04N13/178 ,  H04N13/183 ,  H04N13/243 ,  H04N13/332 ,  H04N13/366 ,  H04N13/344 ,  H04N13/398 ,  H04N13/00

CPC classification number: G06T11/60 ,  G06T19/006 ,  H04N13/156 ,  H04N13/158 ,  H04N13/178 ,  H04N13/183 ,  H04N13/243 ,  H04N13/332 ,  H04N13/344 ,  H04N13/366 ,  H04N13/398 ,  H04N2013/0088

Abstract: Disclosed is a method for displaying a panoramic image by an electronic device. According to an example embodiment of the present disclosure, a method for generating a panoramic image may comprise sensing a direction that a first side surface of the electronic device faces through a sensor included in the electronic device, displaying a first partial image of the panoramic image corresponding to the sensed direction of the first side surface, determining a direction corresponding to the reference view information with respect to the first partial image if information regarding the first partial image differs from reference view information indicating a reference view for the panoramic image, and providing information about the determined direction.

公开(公告)号：US20190068945A1

公开(公告)日：2019-02-28

申请号：US16113967

申请日：2018-08-27

Applicant: CANON KABUSHIKI KAISHA

Inventor： Kazuna Maruyama

IPC: H04N13/117 ,  G06K9/00 ,  G06T15/20

CPC classification number: H04N13/117 ,  A63F13/5255 ,  G06K9/00724 ,  G06K2009/00738 ,  G06T15/20 ,  H04N13/194 ,  H04N13/243 ,  H04N2013/0088

Abstract: An information processing device decides a viewpoint position and generates a virtual viewpoint image based on the decided viewpoint position by using a plurality of images shot by a plurality of imaging apparatuses. The information processing device includes a determining unit configured to determine a scene related to the virtual viewpoint image to be generated, and a deciding unit configured to decide the viewpoint position related to the virtual viewpoint image in the scene determined by the determining unit, based on the scene determined by the determining unit.

公开(公告)号：US10075701B2

公开(公告)日：2018-09-11

申请号：US15046331

申请日：2016-02-17

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss ,  Hector M Medina

IPC: H04N13/189 ,  H04N13/139 ,  H04N13/275 ,  H04N13/194 ,  H04N13/106 ,  H04N13/161 ,  H04N13/279 ,  H04N13/243 ,  H04N19/597 ,  H04N19/172 ,  H04N19/136 ,  H04N21/437 ,  H04N21/472 ,  H04N21/2343 ,  H04N21/2387 ,  H04N5/232 ,  G06T15/04 ,  H04N21/426 ,  H04N21/435 ,  H04N21/44 ,  H04N19/44 ,  H04N21/81 ,  G06T17/20 ,  H04N13/00 ,  H04N13/02

CPC classification number: H04N13/189 ,  G06T15/04 ,  G06T17/20 ,  G06T2215/12 ,  H04N5/23238 ,  H04N13/106 ,  H04N13/139 ,  H04N13/161 ,  H04N13/194 ,  H04N13/243 ,  H04N13/275 ,  H04N13/279 ,  H04N19/136 ,  H04N19/172 ,  H04N19/44 ,  H04N19/597 ,  H04N21/234363 ,  H04N21/2387 ,  H04N21/42653 ,  H04N21/435 ,  H04N21/437 ,  H04N21/44016 ,  H04N21/472 ,  H04N21/8146 ,  H04N21/816 ,  H04N2013/0088

Abstract: Methods and apparatus for using selective resolution reduction on images to be transmitted and/or used by a playback device are described. Prior to transmission one or more images of an environment are captured. Based on image content, motion detection and/or user input a resolution reduction operation is selected and performed. The reduced resolution image is communicated to a playback device along with information indicating a UV map corresponding to the selected resolution allocation that should be used by the playback device for rendering the communicated image. By changing the resolution allocation used and which UV map is used by the playback device different resolution allocations can be made with respect to different portions of the environment while allowing the number of pixels in transmitted images to remain constant. The playback device renders the individual images with the UV map corresponding to the resolution allocation used to generate the individual images.

公开(公告)号：US20180234674A1

公开(公告)日：2018-08-16

申请号：US15889787

申请日：2018-02-06

Applicant: Axon Enterprise, Inc.

Inventor： Patrick W. SMITH ,  Nache D. SHEKARRI ,  William J. SOPER ,  Todd F. BASCHE

IPC: H04N13/282 ,  H04N5/232 ,  H04N13/239 ,  H04N13/00

CPC classification number: H04N13/282 ,  H04N5/2251 ,  H04N5/23238 ,  H04N13/239 ,  H04N2013/0088

Abstract: A system for determining a person's field of view with respect to captured data (e.g., recorded audiovisual data). A multi-camera capture system (e.g. bodycam, vehicular camera, stereoscopic, 360-degree captures system) records audiovisual data of an event. The field of capture of a capture system or a field of capture of captured data combined from multiple capture systems may be greater than the field of view of a person. Facial features (e.g. eyes, ears, nose, and jawlines) of the person may be detected from the captured data. Facial features may be used to determine the field of view of the person with respect to the captured data. Sensors that detect head orientation may be used to determine the field of view of the person with respect to captured data. The field of view of the person may be shown with respect to the captured data when the captured data is played back.

公开(公告)号：US20180220121A1

公开(公告)日：2018-08-02

申请号：US15806251

申请日：2017-11-07

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss ,  Hector M. Medina

IPC: H04N13/189 ,  H04N21/81 ,  H04N13/279 ,  H04N13/106 ,  H04N21/472 ,  H04N21/44 ,  H04N21/437 ,  H04N21/435 ,  H04N21/426 ,  H04N21/2387 ,  H04N21/2343 ,  H04N5/232 ,  G06T17/20 ,  G06T15/04 ,  H04N19/597 ,  H04N19/44 ,  H04N19/172 ,  H04N19/136 ,  H04N13/275 ,  H04N13/243 ,  H04N13/161 ,  H04N13/194 ,  H04N13/139 ,  H04N13/00

CPC classification number: H04N13/189 ,  G06T15/04 ,  G06T17/20 ,  G06T2215/12 ,  H04N5/23238 ,  H04N13/106 ,  H04N13/139 ,  H04N13/161 ,  H04N13/194 ,  H04N13/243 ,  H04N13/275 ,  H04N13/279 ,  H04N19/136 ,  H04N19/172 ,  H04N19/44 ,  H04N19/597 ,  H04N21/234363 ,  H04N21/2387 ,  H04N21/42653 ,  H04N21/435 ,  H04N21/437 ,  H04N21/44016 ,  H04N21/472 ,  H04N21/8146 ,  H04N21/816 ,  H04N2013/0088

Abstract: Methods and apparatus for using selective resolution reduction on images to be transmitted and/or used by a playback device are described. Prior to transmission one or more images of an environment are captured. Based on image content, motion detection and/or user input a resolution reduction operation is selected and performed. The reduced resolution image is communicated to a playback device along with information indicating a UV map corresponding to the selected resolution allocation that should be used by the playback device for rendering the communicated image. By changing the resolution allocation used and which UV map is used by the playback device different resolution allocations can be made with respect to different portions of the environment while allowing the number of pixels in transmitted images to remain constant. The playback device renders the individual images with the UV map corresponding to the resolution allocation used to generate the individual images.

公开(公告)号：US10003740B2

公开(公告)日：2018-06-19

申请号：US14797477

申请日：2015-07-13

Applicant: Futurewei Technologies, Inc.

Inventor： Zhenyu Wu ,  Muhammad Twaha Naqash Ibrahim

IPC: H04N7/18 ,  H04N5/232 ,  H04N5/222 ,  H04N13/00 ,  H04N13/02 ,  G06T3/40 ,  H04N5/247

CPC classification number: H04N5/23238 ,  G06T3/4038 ,  H04N5/222 ,  H04N5/247 ,  H04N13/128 ,  H04N13/243 ,  H04N2013/0081 ,  H04N2013/0088

Abstract: The present disclosure involves systems, software, and computer implemented methods for increasing spatial resolution of panoramic video captured by a camera array. In one example, a method may include identifying a captured image from each camera in a camera array associated with a capture of a panoramic video. The captured images are stitched together to generate at least one combined image and image mode homographies are calculated between the plurality of cameras in the camera array based on the stitching results. A plurality of captured video frames from each camera in the camera array are identified and video mode homographies of the plurality of cameras are determined based on the calculated image mode homographies. The determined video mode homographies are applied to stitch the plurality of captured video frames.