公开(公告)号：US09635338B2

公开(公告)日：2017-04-25

申请号：US14115749

申请日：2012-04-26

Applicant: Shasha Shi ,  Patrick Gioia

Inventor： Shasha Shi ,  Patrick Gioia

IPC: H04N13/00 ,  H04N19/597 ,  H04N19/139 ,  H04N19/172 ,  H04N19/46 ,  H04N19/61 ,  H04N13/02

CPC classification number: H04N13/111 ,  H04N13/161 ,  H04N13/232 ,  H04N19/139 ,  H04N19/172 ,  H04N19/46 ,  H04N19/597 ,  H04N19/61

Abstract: An encoder and method of encoding are provided for encoding at least one integral image representing at least one object in perspective in a scene and including a plurality of elemental images. The method of encoding includes generating a plurality of K sub-images on the basis of the plurality of basic images; arrangement of the sub-images in a predetermined pattern such as to form a multi-view image of the object, the views corresponding respectively to the sub-images; and adaptive compression of the multi-view image formed, as a function of the motion type of the object in the scene.

公开(公告)号：US20170048513A1

公开(公告)日：2017-02-16

申请号：US15236470

申请日：2016-08-14

Applicant: THOMSON LICENSING

Inventor： Benoit VANDAME ,  Mathilde BROSSARD ,  Valter DRAZIC

IPC: H04N13/02 ,  G02B3/00 ,  G02B5/20

CPC classification number: H04N13/232 ,  G02B3/0056 ,  G02B5/201 ,  G06T3/4015 ,  G06T7/557 ,  H04N13/207 ,  H04N13/257

Abstract: A plenoptic camera is proposed having a color filter array positioned on an image sensor with an array of pixels, the color filter array having a first filter with a set of unit elements, each unit element covering M×M pixels of the image sensor, with M an integer such that M≧2. The plenoptic camera further includes a set of micro-lens, each micro-lens delivering a micro-lens image on the image sensor with a diameter equal to p=k×M, with k being an integer greater than or equal to two. The first filter is remarkable in that the set of unit elements comprises an initialization unit element being associated with a matrix ( c m , n ) 0 ≤ m

Abstract translation: 表示niter重新分配（或模式），其中每个系数cm，n与滤波器值相关联，并且其它单位元素与系数设置为c（x + i）mod M的矩阵相关联，（y + j ）mod M，对于图像传感器上的对应像素（x，y，i，j），其中索引x，y涉及图像传感器中的像素的索引，索引i，j涉及微透镜的分度 在一组微型镜头中。

公开(公告)号：US20160360109A1

公开(公告)日：2016-12-08

申请号：US15243737

申请日：2016-08-22

Applicant: THE LIGHTCO INC.

Inventor： Rajiv LAROIA ,  Sapna A. Shroff

IPC: H04N5/232 ,  G02B27/64 ,  G02B27/58 ,  H04N5/225 ,  H04N5/77

CPC classification number: H04N5/23258 ,  G02B27/58 ,  G02B27/646 ,  G06T2207/10052 ,  H04N5/2254 ,  H04N5/2257 ,  H04N5/23238 ,  H04N5/23245 ,  H04N5/23248 ,  H04N5/23277 ,  H04N5/23287 ,  H04N5/247 ,  H04N5/77 ,  H04N13/232

Abstract: Methods and apparatus for processing images captured by a camera device including multiple optical chains, e.g., camera modules, are described. Three, 4, 5 or more optical chains maybe used. Different optical chains capture different images due to different perspectives. Multiple images, e.g., corresponding to different perspectives, are captured during a time period and are combined to generate a composite image. In some embodiments one of the captured images or a synthesized image is used as a reference image during composite image generation. The image used as the reference image is selected to keep the perspective of sequentially generated composite images consistent despite unintentional came movement and/or in accordance with an expected path of travel. Thus, which camera module provides the reference image may vary over time taking into consideration unintended camera movement. Composite image generation may be performed external to the camera device or in the camera device.

公开(公告)号：US20160316144A1

公开(公告)日：2016-10-27

申请号：US15203643

申请日：2016-07-06

Applicant: Lytro, Inc.

Inventor： Jerome Chandra Bhat ,  Brandon Elliott Merle Clarke ,  Graham Butler Myhre ,  Ravi Kiran Nalla ,  Steven David Oliver ,  Tony Yip Pang Poon ,  William D. Houck, II ,  Colvin Pitts ,  Yi-Ren Ng ,  Kurt Akeley

IPC: H04N5/232 ,  G02F1/1337 ,  G02B7/02 ,  H04N13/02 ,  H04N5/225

CPC classification number: H04N5/23245 ,  G02B3/12 ,  G02B7/023 ,  G02B7/08 ,  G02B27/0075 ,  G02F1/1337 ,  H04N5/2254 ,  H04N5/2257 ,  H04N5/23212 ,  H04N5/23229 ,  H04N13/232 ,  H04N13/289 ,  H04N13/296

Abstract: An image capture device, such as a camera, has multiple modes including a light field image capture mode, a conventional 2D image capture mode, and at least one intermediate image capture mode. By changing position and/or properties of the microlens array (MLA) in front of the image sensor, changes in 2D spatial resolution and angular resolution can be attained. In at least one embodiment, such changes can be performed in a continuous manner, allowing a continuum of intermediate modes to be attained.

Abstract translation: 诸如相机的图像捕获设备具有多种模式，包括光场图像捕获模式，常规2D图像捕获模式和至少一种中间图像捕获模式。 通过改变图像传感器前面的微透镜阵列（MLA）的位置和/或性质，可以获得2D空间分辨率和角度分辨率的变化。 在至少一个实施例中，可以以连续的方式执行这种改变，从而允许获得连续的中间模式。

公开(公告)号：US20160316140A1

公开(公告)日：2016-10-27

申请号：US15048369

申请日：2016-02-19

Applicant: Pelican Imaging Corporation

Inventor： Shree Nayar ,  Kartik Venkataraman ,  Bedabrata Pain ,  Dan Lelescu

IPC: H04N5/232 ,  H04N5/225 ,  H04N13/02 ,  H04N9/097

CPC classification number: H04N5/23232 ,  H04N5/2258 ,  H04N5/3415 ,  H04N9/097 ,  H04N13/232 ,  H04N13/271 ,  H04N2013/0081

Abstract: Imager arrays, array camera modules, and array cameras in accordance with embodiments of the invention utilize pixel apertures to control the amount of aliasing present in captured images of a scene. One embodiment includes a plurality of focal planes, control circuitry configured to control the capture of image information by the pixels within the focal planes, and sampling circuitry configured to convert pixel outputs into digital pixel data. In addition, the pixels in the plurality of focal planes include a pixel stack including a microlens and an active area, where light incident on the surface of the microlens is focused onto the active area by the microlens and the active area samples the incident light to capture image information, and the pixel stack defines a pixel area and includes a pixel aperture, where the size of the pixel apertures is smaller than the pixel area.

Abstract translation: 根据本发明的实施例的成像器阵列，阵列相机模块和阵列相机利用像素孔径来控制场景捕获图像中存在的混叠量。 一个实施例包括多个焦平面，控制电路被配置为控制由焦平面内的像素捕获图像信息，以及采样电路，被配置为将像素输出转换为数字像素数据。 此外，多个焦平面中的像素包括包括微透镜和有源区域的像素堆叠，其中入射在微透镜的表面上的光通过微透镜聚焦到有源区域上，并且有源区域将入射光采样到 捕获图像信息，并且像素堆栈限定像素区域并且包括像素孔径，其中像素孔径的尺寸小于像素区域。

公开(公告)号：US09478036B2

公开(公告)日：2016-10-25

申请号：US14668055

申请日：2015-03-25

Applicant: Nokia Technologies Oy

Inventor： Guruaj Gopal Putraya ,  Basavaraja S V ,  Mithun Uliyar ,  Ravi Shenoy

IPC: G06K9/00 ,  G06T7/00 ,  H04N13/02 ,  H04N13/00

CPC classification number: G06T7/0055 ,  G06T7/557 ,  G06T2200/21 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20228 ,  H04N13/232 ,  H04N2013/0081

Abstract: In an example embodiment, a method, apparatus and computer program product are provided. The method includes facilitating receipt of a plenoptic image associated with a scene, the plenoptic image including plenoptic micro-images and being captured by a focused plenoptic camera. The method includes generating plenoptic vectors for the plenoptic micro-images of the plenoptic image, where an individual plenoptic vector is generated for an individual plenoptic micro-image. The method includes assigning disparities for the plenoptic micro-images of the plenoptic image. A disparity for a plenoptic micro-image is assigned by accessing a plurality of subspaces associated with a set of pre-determined disparities, projecting a plenoptic vector for the plenoptic micro-image in the plurality of subspaces, calculating a plurality of residual errors based on projections of the plenoptic vector in the plurality of subspaces, and determining the disparity for the plenoptic micro-image based on a comparison of the plurality of residual errors.

Abstract translation: 在示例实施例中，提供了一种方法，装置和计算机程序产品。 该方法包括促进接收与场景相关联的全景图像，全景图像包括全视觉微图像并被聚焦的全光照相机捕获。 该方法包括为全光图像的全光显微图像产生全光矢量，其中为个体全光显微图像生成个体全光矢量。 该方法包括为全景图像的全景微图像分配差异。 通过访问与一组预定的差异相关联的多个子空间来分配全息微图像的差异，为多个子空间中的全息微图像投射全光矢量，基于 在多个子空间中的全光矢量的投影，以及基于多个残差的比较来确定全光显微图像的视差。

公开(公告)号：US20160307372A1

公开(公告)日：2016-10-20

申请号：US15084326

申请日：2016-03-29

Applicant: Lytro, Inc.

Inventor： Colvin Pitts ,  Yonggang Ha ,  William Houck

IPC: G06T19/00 ,  H04N13/00 ,  H04N13/02

CPC classification number: H04N13/111 ,  H04N5/2254 ,  H04N13/117 ,  H04N13/232 ,  H04N13/243 ,  H04N13/275 ,  H04N13/282 ,  H04N13/344

Abstract: A capture system may capture light-field data representative of an environment for use in virtual reality, augmented reality, and the like. The system may have a plurality of light-field cameras arranged to capture a light-field volume within the environment, and a processor. The processor may use the light-field volume to generate a first virtual view depicting the environment from a first virtual viewpoint. The light-field cameras may be arranged in a tiled array to define a capture surface with a ring-shaped, spherical, or other arrangement. The processor may map the pixels captured by the image sensors to light rays received in the light-field volume, and store data descriptive of the light rays in a coordinate system representative of the light-field volume.

Abstract translation: 捕获系统可捕获代表在虚拟现实中使用的环境的光场数据，增强现实等。 系统可以具有布置成捕获环境内的光场体积的多个光场照相机和处理器。 处理器可以使用光场卷从第一虚拟视点产生描绘环境的第一虚拟视图。 光场照相机可以布置成平铺阵列以限定具有环形，球形或其它布置的捕获表面。 处理器可以将由图像传感器捕获的像素映射到在光场体积中接收的光线，并且将描述光线的数据存储在代表光场体积的坐标系中。

公开(公告)号：US09456134B2

公开(公告)日：2016-09-27

申请号：US14555368

申请日：2014-11-26

Applicant: Pelican Imaging Corporation

Inventor： Kartik Venkataraman ,  Jacques Duparré

IPC: H04N5/232 ,  H04N13/02 ,  H04N17/00 ,  H04N5/369 ,  H04N5/225 ,  H04N5/265 ,  G06T3/40 ,  G06T7/00

CPC classification number: H04N5/23232 ,  G02B3/0056 ,  G02B3/0062 ,  G02B13/02 ,  G06T3/4053 ,  G06T7/557 ,  G06T2207/10052 ,  H04N5/2258 ,  H04N5/23238 ,  H04N5/23296 ,  H04N5/247 ,  H04N5/265 ,  H04N5/3696 ,  H04N13/232 ,  H04N13/243 ,  H04N13/257 ,  H04N17/002 ,  H04N2013/0081

Abstract: Systems and methods for implementing array camera configurations that include a plurality of constituent array cameras, where each constituent array camera provides a distinct field of view and/or a distinct viewing direction, are described. In several embodiments, image data captured by the constituent array cameras is used to synthesize multiple images that are subsequently blended. In a number of embodiments, the blended images include a foveated region. In certain embodiments, the blended images possess a wider field of view than the fields of view of the multiple images.

Abstract translation: 描述了包括多个组成阵列相机的阵列相机配置的系统和方法，其中每个组成阵列相机提供不同的视野和/或不同的观看方向。 在几个实施例中，由组成阵列相机拍摄的图像数据用于合成随后混合的多个图像。 在多个实施例中，混合图像包括移动区域。 在某些实施例中，混合图像具有比多个图像的视野更宽的视野。

公开(公告)号：US20160253839A1

公开(公告)日：2016-09-01

申请号：US15057210

申请日：2016-03-01

Applicant: NextVR Inc.

Inventor： David Cole ,  Alan McKay Moss

IPC: G06T17/20 ,  G06T19/20 ,  G06T17/05

CPC classification number: H04N13/111 ,  G06F3/011 ,  G06F3/012 ,  G06F3/04815 ,  G06K9/00362 ,  G06T17/05 ,  G06T17/20 ,  G06T19/20 ,  G06T2207/30244 ,  G09G3/003 ,  G09G5/001 ,  G09G5/003 ,  G09G5/006 ,  G09G2352/00 ,  G09G2360/02 ,  G09G2360/125 ,  G09G2370/02 ,  H04N13/139 ,  H04N13/161 ,  H04N13/189 ,  H04N13/194 ,  H04N13/204 ,  H04N13/232 ,  H04N13/239 ,  H04N13/271 ,  H04N13/279 ,  H04N13/282 ,  H04N13/344 ,  H04N19/44 ,  H04N21/4345

Abstract: Methods and apparatus for making and using environmental measurements are described. Environmental information captured using a variety of devices is processed and combined to generate an environmental model which is communicated to customer playback devices. A UV map which is used for applying, e.g., wrapping, images onto the environmental model is also provided to the playback devices. A playback device uses the environmental model and UV map to render images which are then displayed to a viewer as part of providing a 3D viewing experience. In some embodiments updated environmental model is generated based on more recent environmental measurements, e.g., performed during the event. The updated environmental model and/or difference information for updating the existing model, optionally along with updated UV map(s), is communicated to the playback devices for use in rendering and playback of subsequently received image content. By communicating updated environmental information improved 3D simulations are achieved.

Abstract translation: 描述了制作和使用环境测量的方法和设备。 使用各种设备捕获的环境信息被处理并组合以产生传达给客户回放设备的环境模型。 用于将例如将图像包裹在环境模型上的UV图也被提供给回放装置。 播放设备使用环境模型和UV映射来渲染图像，然后将其显示给观看者，作为提供3D观看体验的一部分。 在一些实施例中，基于更近期的环境测量（例如，在事件期间执行）来生成更新的环境模型。 用于更新现有模型的更新的环境模型和/或差异信息（可选地与更新的UV图）一起被传送到回放设备，以用于随后接收的图像内容的呈现和回放。 通过传达更新的环境信息，实现了改进的3D模拟。

公开(公告)号：US20160253795A1

公开(公告)日：2016-09-01

申请号：US15051550

申请日：2016-02-23

Applicant: NextVR, Inc.

Inventor： David Michael Cole ,  Alan McKay Moss ,  Hector Medina

IPC: G06T7/00 ,  G06T19/00 ,  G02B27/01 ,  G06T15/04

CPC classification number: G06T7/80 ,  G02B27/0172 ,  G02B2027/0138 ,  G02B2027/014 ,  G06T5/006 ,  G06T7/85 ,  G06T11/001 ,  G06T11/40 ,  G06T15/04 ,  G06T19/003 ,  G06T19/006 ,  G06T2207/10021 ,  H04N13/232 ,  H04N13/246 ,  H04N13/275 ,  H04N21/41407 ,  H04N21/4402 ,  H04N21/4854 ,  H04N21/816

Abstract: Camera and/or lens calibration information is generated as part of a calibration process in video systems including 3-dimensional (3D) immersive content systems. The calibration information can be used to correct for distortions associated with the source camera and/or lens. A calibration profile can include information sufficient to allow the system to correct for camera and/or lens distortion/variation. This can be accomplished by capturing a calibration image of a physical 3D object corresponding to the simulated 3D environment, and creating the calibration profile by processing the calibration image. The calibration profile can then be used to project the source content directly into the 3D viewing space while also accounting for distortion/variation, and without first translating into an intermediate space (e.g., a rectilinear space) to account for lens distortion.

Abstract translation: 摄像机和/或镜头校准信息作为包括三维（3D）沉浸式内容系统在内的视频系统中的校准过程的一部分生成。 校准信息可用于校正与源摄像机和/或镜头相关的失真。 校准曲线可以包括足以允许系统校正摄像机和/或镜头失真/变化的信息。 这可以通过捕获与模拟3D环境相对应的物理3D物体的校准图像来实现，并且通过处理校准图像来创建校准轮廓。 然后可以将校准曲线用于将源内容投影到3D观看空间中，同时也考虑到失真/变化，并且不首先将其转换成中间空间（例如，直线空间）以考虑透镜失真。