公开(公告)号：US20050078116A1

公开(公告)日：2005-04-14

申请号：US10815141

申请日：2004-03-31

申请人： Peter-Pike Sloan ,  Yi-Ren Ng ,  John Snyder

发明人： Peter-Pike Sloan ,  Yi-Ren Ng ,  John Snyder

IPC分类号： F21V9/00 ,  G06F15/16

CPC分类号： G06T15/55 ,  G06T15/50

摘要： The present invention is directed to a enhanced Precomputed Radiance Transfer (PRT) system employing an algorithm to compute a PRT signal over a surface mesh and subdividing facets of the mesh to increase the number of surface vertices such that the spatial variation of the transfer signal is resolved sufficiently everywhere on the surface. The method of this system ensures that radiance transfer shading produces colors of sufficient accuracy all over the surface. In certain embodiments, transfer is computed only at surface vertices, although this does result in a certain amount of acceptable aliasing and blurring of surface lighting detail in regions where the tessellation is too coarse. Furthermore, the method comprises a spatial and density sampling techniques that measures the transfer signal to a desirable appropriate resolution while minimizing aliasing. Once computed, the signal is represented as compactly as possible to minimize storage and runtime computation requirements.

摘要翻译： 本发明涉及一种增强的预计算辐射传输（PRT）系统，该系统采用一种算法来在表面网格上计算PRT信号并细分网格面，以增加表面顶点的数量，使得传输信号的空间变化为 在表面上的任何地方充分解决。 该系统的方法确保辐射传递阴影在整个表面上产生足够精确的颜色。 在某些实施例中，仅在表面顶点处计算传输，尽管这确实导致在细分太粗糙的区域中的一定量的可接受的混叠和模糊表面照明细节。 此外，该方法包括空间和密度采样技术，其将传输信号测量到期望的适当分辨率，同时最小化混叠。 一旦计算，信号尽可能紧凑地表示，以最小化存储和运行时计算要求。

公开(公告)号：US20050080602A1

公开(公告)日：2005-04-14

申请号：US10815140

申请日：2004-03-31

申请人： John Snyder ,  Peter-Pike Sloan ,  Yi-Ren Ng

发明人： John Snyder ,  Peter-Pike Sloan ,  Yi-Ren Ng

IPC分类号： G06F17/10 ,  G06T15/50

CPC分类号： G06T15/506

摘要： The present invention is directed to systems and methods for all-frequency relighting by representing low frequencies of lighting with spherical harmonics and approximate the residual high-frequency energy with point lights. One such embodiment renders low-frequencies with a precomputed radiance transfer (PRT) technique (which requires only a moderate amount of precomputation and storage), while the higher-frequencies are rendered with on-the-fly techniques such as shadow maps and shadow volumes. In addition, various embodiments are directed to a systems and methods for decomposing the lighting into harmonics and sets of point lights. Various alternative embodiments are directed to systems and methods for characterizing the types of environments for which the described decomposition is a viable technique in terms of speed (efficiency) versus quality (realism).

摘要翻译： 本发明涉及通过用球面谐波表示低频照明并用点光近似残余高频能量来进行全频重启的系统和方法。 一个这样的实施例使用预计算辐射传输（PRT）技术（其仅需要适度的预先计算和存储）来实现低频，而较高频率是使用诸如阴影图和阴影体积的飞行技术来渲染的 。 此外，各种实施例涉及用于将照明分解成谐波和点光源的系统和方法。 各种替代实施例涉及用于表征在速度（效率）与质量（实际）方面所描述的分解是可行技术的环境类型的系统和方法。

公开(公告)号：US08570426B2

公开(公告)日：2013-10-29

申请号：US12622655

申请日：2009-11-20

申请人： Colvin Pitts ,  Yi-Ren Ng

发明人： Colvin Pitts ,  Yi-Ren Ng

IPC分类号： H04N5/232 ,  G06K9/40

CPC分类号： H04N5/23212 ,  H04N7/01 ,  H04N13/0207 ,  H04N13/207

摘要： Certain systems and methods are directed to acquiring, generating, manipulating and/or editing (for example, focusing or refocusing) refocusable video data, information, images and/or frames. The refocusable video data, information, images and/or frames may be light field video data, information, images and/or frames, that may be focused and/or re-focused after acquisition or recording of such video data, information, images and/or frames. In one aspect, a method of generating video data of a scene using a video acquisition device which acquires refocusable light field video data is disclosed, the method comprising (a) acquiring first refocusable light field video data of a scene, (b) storing first refocusable video data which is representative of the first refocusable light field video data, (c) acquiring second refocusable light field video data of the scene after acquiring the first refocusable light field video data, (d) determining a first virtual focus parameter (for example, a virtual focus depth) using the second refocusable light field video data, (e) generating first video data using the stored first refocusable video data and the first virtual focus parameter, wherein the first video data includes a focus depth that is different from an optical focus depth of the first refocusable light field video data, and (f) outputting the first video data to, for example, memory, a video display, processing circuitry, and/or a recording device.

摘要翻译： 某些系统和方法旨在获取，生成，操纵和/或编辑（例如，聚焦或重新聚焦）可重新聚焦的视频数据，信息，图像和/或帧。 可重新聚焦的视频数据，信息，图像和/或帧可以是在获取或记录这样的视频数据，信息，图像之后可以聚焦和/或重新聚焦的光场视频数据，信息，图像和/或帧， /或帧。 一方面，公开了一种使用获取可重构的光场视频数据的视频采集装置产生场景的视频数据的方法，所述方法包括：（a）获取场景的第一可重新聚焦的光场视频数据，（b）首先存储 （c）在获取第一可再聚焦光场视频数据之后获取场景的第二可再聚焦光场视频数据，（d）确定第一虚拟焦点参数（例如，第一可重定位光场视频数据） ，虚拟焦点深度），（e）使用所存储的第一可重新聚焦的视频数据和第一虚拟焦点参数来生成第一视频数据，其中第一视频数据包括不同于 第一可再聚焦光场视频数据的光学聚焦深度，以及（f）将第一视频数据输出到例如存储器，视频显示器，处理电路和 /或记录装置。

公开(公告)号：US08446516B2

公开(公告)日：2013-05-21

申请号：US13563513

申请日：2012-07-31

申请人： Colvin Pitts ,  Yi-Ren Ng

发明人： Colvin Pitts ,  Yi-Ren Ng

IPC分类号： H04N5/232

CPC分类号： H04N5/23212 ,  H04N7/01 ,  H04N13/0207 ,  H04N13/207

摘要： Systems and methods are directed to acquiring, generating, manipulating and/or editing refocusable video data/frames. The refocusable video frames may be light field video frames that may be focused and/or refocused after acquisition or recording of such video frames. In one aspect, a method comprises: selecting a first key frame, wherein the first key frame corresponds to one of a plurality of refocusable light field video frames; selecting a second key frame, wherein the second key frame corresponds to one of the plurality of refocusable light field video frames which is temporally spaced apart from the first key frame such that a plurality of refocusable light field video frames are temporally disposed between the first and the second key frames; determining virtual focus parameters for the first key frame and the second key frame; and generating video data.

摘要翻译： 系统和方法旨在获取，生成，操纵和/或编辑可重新聚焦的视频数据/帧。 可重新聚焦的视频帧可以是在获取或记录这样的视频帧之后可以聚焦和/或重新聚焦的光场视频帧。 一方面，一种方法包括：选择第一关键帧，其中所述第一关键帧对应于多个可重新聚焦的光场视频帧之一; 选择第二关键帧，其中所述第二关键帧对应于与所述第一关键帧在时间上间隔开的所述多个可重整的光场视频帧中的一个，使得多个可重新聚焦的光场视频帧被时间上设置在所述第一和第 第二关键帧; 确定所述第一关键帧和所述第二关键帧的虚拟焦点参数; 并产生视频数据。

公开(公告)号：US08279325B2

公开(公告)日：2012-10-02

申请号：US12622673

申请日：2009-11-20

申请人： Colvin Pitts ,  Yi-Ren Ng

发明人： Colvin Pitts ,  Yi-Ren Ng

IPC分类号： G03B13/00

CPC分类号： H04N5/23212 ,  H04N7/01 ,  H04N13/0207 ,  H04N13/207

摘要： Systems and methods directed to acquiring, generating, manipulating and/or editing refocusable video data/frames. The refocusable video frames may be light field video frames that may be focused and/or re-focused after acquisition or recording of such video frames. In one aspect, method comprises: selecting a first key frame, wherein the first key frame corresponds to one of a plurality of refocusable light field video frames, selecting a second key frame, wherein the second key frame corresponds to one of the plurality of refocusable light field video frames which is temporally spaced apart from the first key frame such that a plurality of refocusable light field video frames are temporally disposed between the first and the second key frames, determining a virtual focus parameters for the first key frame and the second key frame, and generating first video data.

摘要翻译： 旨在获取，生成，操纵和/或编辑可重新聚焦的视频数据/帧的系统和方法。 可再结合的视频帧可以是在获取或记录这样的视频帧之后可以聚焦和/或重新聚焦的光场视频帧。 一方面，方法包括：选择第一关键帧，其中所述第一关键帧对应于多个可重新聚焦的光场视频帧中的一个，选择第二关键帧，其中所述第二关键帧对应于所述多个可重新对准的一个 光场视频帧在时间上与第一关键帧间隔开，使得多个可重新聚焦的光场视频帧在时间上设置在第一和第二关键帧之间，确定第一关键帧和第二关键帧的虚拟聚焦参数 帧，并生成第一视频数据。

公开(公告)号：US20130342526A1

公开(公告)日：2013-12-26

申请号：US13533319

申请日：2012-06-26

申请人： Yi-Ren Ng ,  Eric Cheng ,  Chia-Kai Liang ,  Kayvon Fatahalian ,  David John Evans ,  Kira Wampler ,  Kristen Berman ,  Kurt Barton Akeley

发明人： Yi-Ren Ng ,  Eric Cheng ,  Chia-Kai Liang ,  Kayvon Fatahalian ,  David John Evans ,  Kira Wampler ,  Kristen Berman ,  Kurt Barton Akeley

IPC分类号： G06T15/00

CPC分类号： G06T15/005 ,  G06T15/205 ,  G06T19/00 ,  G06T2200/21

摘要： According to various embodiments of the invention, a system and method are provided for enabling interaction with, manipulation of, and control of depth-assigned content in depth-enhanced pictures. Depth-assigned content can be assigned to a specified depth value. When a depth-enhanced picture is refocused at a focus depth substantially different from the specified assigned depth value, the depth-assigned content may be omitted, grayed out, blurred, or otherwise visually distinguished. In this manner, content associated with an in-focus image element can be visually distinguished from content associated with an out-of-focus image element. For example, in at least one embodiment, depth-assigned content is visible only when an image element associated with the content is in focus (or nearly in focus). According to various embodiments of the invention, many different types of interactions are facilitated among depth-assigned content, depth-enhanced pictures, and other content.

摘要翻译： 根据本发明的各种实施例，提供了一种系统和方法，用于使得能够与深度增强的图像中的深度分配的内容进行交互，操纵和控制。 深度分配的内容可以分配给指定的深度值。 当深度增强图像被重新聚焦在与指定的指定深度值基本上不同的焦点深度时，可以省略深度分配的内容，变灰，模糊或以其他方式视觉区分。 以这种方式，与对焦图像元素相关联的内容可以与视频区分与与离焦图像元素相关联的内容。 例如，在至少一个实施例中，只有当与内容相关联的图像元素处于焦点（或几乎聚焦）时，深度分配的内容才可见。 根据本发明的各种实施例，在深度分配的内容，深度增强的图片和其他内容之间促进了许多不同类型的交互。

公开(公告)号：US20120327222A1

公开(公告)日：2012-12-27

申请号：US13603275

申请日：2012-09-04

申请人： Yi-Ren Ng ,  Colvin Pitts ,  Timothy Knight

发明人： Yi-Ren Ng ,  Colvin Pitts ,  Timothy Knight

IPC分类号： H04N7/18

CPC分类号： H04N5/2254 ,  G02B7/282 ,  G02B27/0075 ,  G06T2200/21 ,  G06T2207/10052 ,  H04N5/225 ,  H04N5/23212

摘要： A light field data acquisition device includes optics and a light field sensor to acquire light field image data of a scene. In at least one embodiment, the light field sensor is located at a substantially fixed, predetermined distance relative to the focal point of the optics. In response to user input, the light field acquires the light field image data of the scene, and a storage device stores the acquired data. Such acquired data can subsequently be used to generate a plurality of images of the scene using different virtual focus depths.

摘要翻译： 光场数据采集装置包括用于获取场景的光场图像数据的光学器件和光场传感器。 在至少一个实施例中，光场传感器相对于光学器件的焦点位于基本上固定的预定距离处。 响应于用户输入，光场获取场景的光场图像数据，并且存储装置存储获取的数据。 随后可以使用这样获取的数据来使用不同的虚拟聚焦深度来生成场景的多个图像。

公开(公告)号：US20120294590A1

公开(公告)日：2012-11-22

申请号：US13563513

申请日：2012-07-31

申请人： Colvin Pitts ,  Yi-Ren Ng

发明人： Colvin Pitts ,  Yi-Ren Ng

IPC分类号： H04N13/00 ,  H04N5/93

CPC分类号： H04N5/23212 ,  H04N7/01 ,  H04N13/0207 ,  H04N13/207

摘要： Systems and methods are directed to acquiring, generating, manipulating and/or editing refocusable video data/frames. The refocusable video frames may be light field video frames that may be focused and/or refocused after acquisition or recording of such video frames. In one aspect, a method comprises: selecting a first key frame, wherein the first key frame corresponds to one of a plurality of refocusable light field video frames; selecting a second key frame, wherein the second key frame corresponds to one of the plurality of refocusable light field video frames which is temporally spaced apart from the first key frame such that a plurality of refocusable light field video frames are temporally disposed between the first and the second key frames; determining virtual focus parameters for the first key frame and the second key frame; and generating video data.

摘要翻译： 系统和方法旨在获取，生成，操纵和/或编辑可重新聚焦的视频数据/帧。 可重新聚焦的视频帧可以是在获取或记录这样的视频帧之后可以聚焦和/或重新聚焦的光场视频帧。 一方面，一种方法包括：选择第一关键帧，其中所述第一关键帧对应于多个可重新聚焦的光场视频帧之一; 选择第二关键帧，其中所述第二关键帧对应于与所述第一关键帧在时间上间隔开的所述多个可重整的光场视频帧中的一个，使得多个可重新聚焦的光场视频帧被时间上设置在所述第一和第 第二关键帧; 确定所述第一关键帧和所述第二关键帧的虚拟焦点参数; 并产生视频数据。

公开(公告)号：US07723662B2

公开(公告)日：2010-05-25

申请号：US12089371

申请日：2006-10-06

申请人： Marc S. Levoy ,  Yi-Ren Ng ,  Mark A. Horowitz

发明人： Marc S. Levoy ,  Yi-Ren Ng ,  Mark A. Horowitz

IPC分类号： H01L27/00

CPC分类号： G02B21/361 ,  G02B21/367 ,  G02B27/0075

摘要： Light-field microscopy is facilitated using an approach to image computation. In connection with an example embodiment, a subject (e.g., 105) is imaged by passing light from the subject through a microlens array (e.g., 120) to a photosensor array (e.g., 130) to simultaneously detect light from the subject that is passed through different directions to different locations. In certain embodiments, information from the detected light is used to compute refocused images, perspective images and/or volumetric datasets, from a single-shot photograph.

摘要翻译： 使用图像计算的方法来促进光场显微镜。 结合示例实施例，通过将来自受试者的光通过微透镜阵列（例如，120）传递到光电传感器阵列（例如，130）来对被摄体（例如，105）进行成像，以同时检测来自被摄体的光 通过不同的方向到不同的位置。 在某些实施例中，来自检测到的光的信息用于从单次照片计算重聚焦图像，透视图像和/或体积数据集。

公开(公告)号：US20090128669A1

公开(公告)日：2009-05-21

申请号：US12278708

申请日：2007-02-06

申请人： Yi-Ren Ng ,  Patrick M. Hanrahan ,  Mark A. Horowitz ,  Marc S. Levoy

发明人： Yi-Ren Ng ,  Patrick M. Hanrahan ,  Mark A. Horowitz ,  Marc S. Levoy

IPC分类号： H04N5/217

CPC分类号： G02B7/08 ,  G02B3/0006 ,  G02B3/0056 ,  G02B3/0075 ,  G02B27/0075 ,  H04N5/2253 ,  H04N5/23212

摘要： Digital images are computed using an approach for correcting lens aberration. According to an example embodiment of the present invention, a digital imaging arrangement implements microlenses to direct light to photosensors that detect the light and generate data corresponding to the detected light. The generated data is used to compute an output image, where each output image pixel value corresponds to a selective weighting and summation of a subset of the detected photosensor values. The weighting is a function of characteristics of the imaging arrangement. In some applications, the weighting reduces the contribution of data from photosensors that contribute higher amounts of optical aberration to the corresponding output image pixel.

摘要翻译： 使用用于校正透镜像差的方法来计算数字图像。 根据本发明的示例性实施例，数字成像装置实现微透镜以将光引导到检测光的光电传感器并产生对应于检测到的光的数据。 生成的数据用于计算输出图像，其中每个输出图像像素值对应于检测到的光电传感器值的子集的选择性加权和求和。 加权是成像装置的特征的函数。 在一些应用中，加权减少了对相应的输出图像像素贡献更大量的光学像差的光电传感器的数据的贡献。