公开(公告)号：US20150362743A1

公开(公告)日：2015-12-17

申请号：US14836212

申请日：2015-08-26

Applicant: Douglas Lanman ,  Matthew Hirsch ,  Yun Hee Kim ,  Szymon Jakubczak ,  Ramesh Raskar

Inventor： Douglas Lanman ,  Matthew Hirsch ,  Yun Hee Kim ,  Szymon Jakubczak ,  Ramesh Raskar

IPC: G02B27/22 ,  H04N13/04

CPC classification number: G02B27/2214 ,  G02B27/225 ,  G02B27/2264 ,  H04N13/307 ,  H04N13/315 ,  H04N13/32

Abstract: In exemplary implementations of this invention, two LCD screens display a multi-view 3D image that has both horizontal and vertical parallax, and that does not require a viewer to wear any special glasses. Each pixel in the LCDs can take on any value: the pixel can be opaque, transparent, or any shade between. For regions of the image that are adjacent to a step function (e.g., a depth discontinuity) and not adjacent to a sharp corner, the screens display local parallax barriers comprising many small slits. The barriers and the slits tend to be oriented perpendicular to the local angular gradient of the target light field. In some implementations, the display is optimized to seek to minimize the Euclidian distance between the desired light field and the actual light field that is produced. Weighted, non-negative matrix factorization (NMF) is used for this optimization.

Abstract translation: 在本发明的示例性实施方案中，两个LCD屏幕显示具有水平和垂直视差的多视图3D图像，并且不需要观看者佩戴任何特殊眼镜。 LCD中的每个像素可以承担任何值：像素可以是不透明的，透明的或任何阴影之间。 对于邻近阶梯函数（例如，深度不连续）并且不与锐角相邻的图像的区域，屏幕显示包括许多小缝隙的局部视差屏障。 障碍物和狭缝倾向于垂直于目标光场的局部角度梯度取向。 在一些实施方案中，显示器被优化以寻求使期望的光场和所产生的实际光场之间的欧几里德距离最小化。 加权非负矩阵因子分解（NMF）用于此优化。

公开(公告)号：US09146317B2

公开(公告)日：2015-09-29

申请号：US13479185

申请日：2012-05-23

Applicant: Rohit Pandharkar ,  Andreas Velten ,  Ramesh Raskar

Inventor： Rohit Pandharkar ,  Andreas Velten ,  Ramesh Raskar

IPC: G01S17/58 ,  G01S17/89

CPC classification number: G01S17/58 ,  G01S17/89

Abstract: In exemplary implementations of this invention, a time of flight camera (ToF camera) can estimate the location, motion and size of a hidden moving object, even though (a) the hidden object cannot be seen directly (or through mirrors) from the vantage point of the ToF camera (including the camera's illumination source and sensor), and (b) the object is in a visually cluttered environment. The hidden object is a NLOS (non-line-of-sight) object. The time of flight camera comprises a streak camera and a laser. In these exemplary implementations, the motion and absolute locations of NLOS moving objects in cluttered environments can be estimated through tertiary reflections of pulsed illumination, using relative time differences of arrival at an array of receivers. Also, the size of NLOS moving objects can be estimated by backprojecting extremas of NLOS moving object time responses.

Abstract translation: 在本发明的示例性实施方式中，飞行时间相机（ToF相机）可以估计隐藏的移动物体的位置，运动和尺寸，即使（a）从有利位置看不到直接（或通过反射镜）隐藏的物体 ToF相机（包括相机的照明源和传感器）的点，（b）物体在视觉上混乱的环境中。 隐藏的对象是一个NLOS（非视距）对象。 飞行时间相机包括条纹相机和激光。 在这些示例性实施方案中，可以使用到达接收器阵列的相对时间差异，通过脉冲照明的三次反射来估计NLOS移动物体在杂乱环境中的运动和绝对位置。 此外，NLOS移动对象的大小可以通过反向投影NLOS移动对象时间响应的极值来估计。

公开(公告)号：US09143678B2

公开(公告)日：2015-09-22

申请号：US13102812

申请日：2011-05-06

Applicant: Byung-Kwan Park ,  Ghulam Ahmed Kirmani ,  Ramesh Raskar ,  Rohit Pandharkar

Inventor： Byung-Kwan Park ,  Ghulam Ahmed Kirmani ,  Ramesh Raskar ,  Rohit Pandharkar

IPC: H04N5/228 ,  H04N5/232 ,  H04N5/225 ,  G06T7/00 ,  H04N13/00

CPC classification number: H04N5/23229 ,  G06T7/557 ,  G06T2200/21 ,  G06T2207/10052 ,  G06T2207/20048 ,  G06T2207/20212 ,  H04N5/2254 ,  H04N2013/0081

Abstract: Provided are an apparatus and method for processing a light field image that is acquired and processed using a mask to spatially modulate a light field. The apparatus includes a lens, a mask to spatially modulate 4D light field data of a scene passing through the lens to include wideband information on the scene, a sensor to detect a 2D image corresponding to the spatially modulated 4D light field data, and a data processing unit to recover the 4D light field data from the 2D image to generate an all-in-focus image.

Abstract translation: 提供了一种用于处理使用掩模获取和处理的光场图像以在空间上调制光场的装置和方法。 该装置包括透镜，用于对通过透镜的场景的4D光场数据进行空间调制以包括场景上的宽带信息的掩模，用于检测对应于空间调制的4D光场数据的2D图像的传感器和数据 处理单元，以从2D图像恢复4D光场数据，以产生全焦点图像。

公开(公告)号：US20150035880A1

公开(公告)日：2015-02-05

申请号：US14451666

申请日：2014-08-05

Applicant: Felix Heide ,  Gordon Wetzstein ,  James Gregson ,  Ramesh Raskar ,  Wolfgang Heidrich

Inventor： Felix Heide ,  Gordon Wetzstein ,  James Gregson ,  Ramesh Raskar ,  Wolfgang Heidrich

IPC: G09G3/36

CPC classification number: G09G3/36 ,  G09G2300/023 ,  G09G2340/0457

Abstract: In exemplary implementations of this invention, light from a backlight is transmitted through two stacked LCDs and then through a diffuser. The front side of the diffuser displays a time-varying sequence of 2D images. Processors execute an optimization algorithm to compute optimal pixel states in the first and second LCDs, respectively, such that for each respective image in the sequence, the optimal pixel states minimize, subject to one or more constraints, a difference between a target image and the respective image. The processors output signals to control actual pixel states in the LCDs, based on the computed optimal pixel states. The 2D images displayed by the diffuser have a higher spatial resolution than the native spatial resolution of the LCDs. Alternatively, the diffuser may be switched off, and the device may display either (a) 2D images with a higher dynamic range than the LCDs, or (b) an automultiscopic display.

Abstract translation: 在本发明的示例性实施方案中，来自背光的光通过两个堆叠的LCD传输，然后通过扩散器传输。 漫射器的前侧显示2D图像的时变序列。 处理器执行优化算法以分别计算第一和第二LCD中的最佳像素状态，使得对于序列中的每个相应图像，最佳像素状态在受到一个或多个约束的情况下最小化目标图像与 各自的图像。 处理器基于所计算的最佳像素状态输出信号以控制LCD中的实际像素状态。 由漫射器显示的2D图像具有比LCD的原始空间分辨率更高的空间分辨率。 或者，可以关闭扩散器，并且该装置可以显示（a）具有比LCD更高的动态范围的2D图像，或（b）自动显微镜显示器。

公开(公告)号：US08848006B2

公开(公告)日：2014-09-30

申请号：US13736769

申请日：2013-01-08

Applicant: Gordon Wetzstein ,  Douglas Lanman ,  Matthew Hirsch ,  Ramesh Raskar

Inventor： Gordon Wetzstein ,  Douglas Lanman ,  Matthew Hirsch ,  Ramesh Raskar

IPC: G09G5/10 ,  G09G3/34

CPC classification number: G09G3/3406 ,  G09G3/003 ,  G09G3/36 ,  G09G2300/023 ,  G09G2310/0235 ,  G09G2340/00 ,  H04N13/305 ,  H04N13/307 ,  H04N13/31 ,  H04N13/351 ,  H04N13/354 ,  H04N13/366

Abstract: In exemplary implementations of this invention, an automultiscopic display device includes (1) one or more spatially addressable, light attenuating layers, and (2) a controller which is configured to perform calculations to control the device. In these calculations, tensors provide sparse, memory-efficient representations of a light field. The calculations include using weighted nonnegative tensor factorization (NTF) to solve an optimization problem. The NTF calculations can be sufficiently efficient to achieve interactive refresh rates. Either a directional backlight or a uniform backlight may be used. For example, the device may have (1) a high resolution LCD in front, and (2) a low resolution directional backlight. Or, for example, the device may have a uniform backlight and three or more LCD panels. In these examples, all of the LCDs and the directional backlight (if applicable) may be time-multiplexed.

Abstract translation: 在本发明的示例性实施方案中，自动不透明显示装置包括（1）一个或多个空间可寻址的光衰减层，以及（2）被配置为执行计算以控制该装置的控制器。 在这些计算中，张量提供光场的稀疏，记忆效能表示。 计算包括使用加权非负张力因子分解（NTF）来解决优化问题。 NTF计算可以有效地实现交互式刷新率。 可以使用定向背光或均匀背光。 例如，该装置可以具有（1）前面的高分辨率LCD，以及（2）低分辨率定向背光。 或者，例如，该设备可以具有均匀的背光和三个或更多个LCD面板。 在这些示例中，所有LCD和定向背光（如果适用）可以被时分复用。

公开(公告)号：US08783871B2

公开(公告)日：2014-07-22

申请号：US13577880

申请日：2011-04-22

Applicant: Vitor Pamplona ,  Manuel Menezes de Oliveira Neto ,  Ankit Mohan ,  Ramesh Raskar

Inventor： Vitor Pamplona ,  Manuel Menezes de Oliveira Neto ,  Ankit Mohan ,  Ramesh Raskar

IPC: A61B3/02 ,  A61B3/00

CPC classification number: A61B3/032

Abstract: In exemplary implementations, this invention is a tool for subjective assessment of the visual acuity of a human eye. A microlens or pinhole array is placed over a high-resolution display. The eye is brought very near to the device. Patterns are displayed on the screen under some of the lenslets or pinholes. Using interactive software, a user causes the patterns that the eye sees to appear to be aligned. The software allows the user to move the apparent position of the patterns. This apparent motion is achieved by pre-warping the position and angle of the ray-bundles exiting the lenslet display. As the user aligns the apparent position of the patterns, the amount of pre-warping varies. The amount of pre-warping required in order for the user to see what appears to be a single, aligned pattern indicates the lens aberration of the eye.

Abstract translation: 在示例性实施方式中，本发明是用于主观评估人眼视力的工具。 微透镜或针孔阵列放置在高分辨率显示器上。 眼睛非常靠近设备。 图案显示在屏幕上的一些小透镜或针孔下。 使用交互式软件，用户会使眼睛看到的图案对齐。 该软件允许用户移动图案的明显位置。 这种明显的运动是通过预处理离开小透镜显示器的射线束的位置和角度来实现的。 当用户对准图案的表观位置时，预​​翘曲的量变化。 为了使用户看到似乎是单个对齐的图案所需的预翘曲量指示眼睛的镜片像差。

公开(公告)号：US08651678B2

公开(公告)日：2014-02-18

申请号：US13689631

申请日：2012-11-29

Applicant: Douglas Lanman ,  Gordon Wetzstein ,  Matthew Hirsch ,  Wolfgang Heidrich ,  Ramesh Raskar

Inventor： Douglas Lanman ,  Gordon Wetzstein ,  Matthew Hirsch ,  Wolfgang Heidrich ,  Ramesh Raskar

IPC: F21V9/14

CPC classification number: G02B27/22 ,  F21V9/14 ,  G02B27/225 ,  G09G3/003 ,  G09G2300/023 ,  G09G2300/0447 ,  G09G2310/0235 ,  H04N13/32 ,  H04N13/324

Abstract: In exemplary implementations of this invention, a flat screen device displays a 3D scene. The 3D display may be viewed by a person who is not wearing any special glasses. The flat screen device displays dynamically changing 3D imagery, with a refresh rate so fast that the device may be used for 3D movies or for interactive, 3D display. The flat screen device comprises a stack of LCD layers with two crossed polarization filters, one filter at each end of the stack. One or more processors control the voltage at each pixel of each LCD layer, in order to control the polarization state rotation induced in light at that pixel. The processor employs an algorithm that models each LCD layer as a spatially-controllable polarization rotator, rather than a conventional spatial light modulator that directly attenuates light. Color display is achieved using field sequential color illumination with monochromatic LCDs.

Abstract translation: 在本发明的示例性实现中，平面屏幕设备显示3D场景。 3D显示器可以由没有佩戴任何特殊眼镜的人员来观看。 平面屏幕设备显示动态变化的3D图像，刷新速度非常快，以至于该设备可用于3D电影或交互式3D显示。 平面屏幕装置包括具有两个交叉偏振滤光器的LCD层的堆叠，在堆叠的每个端部具有一个滤光器。 一个或多个处理器控制每个LCD层的每个像素处的电压，以便控制在该像素处的光中引起的偏振状态旋转。 处理器采用将每个LCD层建模为空间可控的偏振旋转器的算法，而不是直接衰减光的常规空间光调制器。 使用具有单色LCD的场顺序彩色照明来实现彩色显示。

公开(公告)号：US20130208241A1

公开(公告)日：2013-08-15

申请号：US13766751

申请日：2013-02-13

Applicant: Matthew Everett Lawson ,  Jason Boggess ,  Siddharth Khullar ,  Ramesh Raskar

Inventor： Matthew Everett Lawson ,  Jason Boggess ,  Siddharth Khullar ,  Ramesh Raskar

IPC: A61B3/14

CPC classification number: A61B3/14 ,  A61B3/0025 ,  A61B3/0091 ,  A61B3/12 ,  G06K9/00604

Abstract: In exemplary implementations, this invention comprises apparatus for retinal self-imaging. Visual stimuli help the user self-align his eye with a camera. Bi-ocular coupling induces the test eye to rotate into different positions. As the test eye rotates, a video is captured of different areas of the retina. Computational photography methods process this video into a mosaiced image of a large area of the retina. An LED is pressed against the skin near the eye, to provide indirect, diffuse illumination of the retina. The camera has a wide field of view, and can image part of the retina even when the eye is off-axis (when the eye's pupillary axis and camera's optical axis are not aligned). Alternately, the retina is illuminated directly through the pupil, and different parts of a large lens are used to image different parts of the retina. Alternately, a plenoptic camera is used for retinal imaging.

Abstract translation: 在示例性实施方式中，本发明包括用于视网膜自我成像的装置。 视觉刺激帮助用户用相机自行调整眼睛。 双眼耦合引起测试眼旋转到不同的位置。 当测试眼旋转时，捕获视网膜的不同区域的视频。 计算摄影方法将该视频处理成视网膜大面积的马赛克图像。 LED被压在眼睛附近的皮肤上，以提供视网膜的间接的漫射照明。 相机具有宽视野，即使眼睛离轴（眼睛的瞳孔轴和相机的光轴未对齐），也可以对视网膜的一部分进行成像。 或者，视网膜直接通过瞳孔照射，并且使用大透镜的不同部分来对视网膜的不同部分进行成像。 或者，全视相机用于视网膜成像。

公开(公告)号：US20110017826A1

公开(公告)日：2011-01-27

申请号：US12838304

申请日：2010-07-16

Applicant: Ankit Mohan ,  Ramesh Raskar ,  Shinsaku Hiura ,  Quinn Smithwick ,  Grace Woo

Inventor： Ankit Mohan ,  Ramesh Raskar ,  Shinsaku Hiura ,  Quinn Smithwick ,  Grace Woo

IPC: G06K7/10

CPC classification number: G06K7/10722 ,  G06K7/10831

Abstract: In an illustrative implementation of this invention, an optical pattern that encodes binary data is printed on a transparency. For example, the pattern may comprise data matrix codes. A lenslet is placed at a distance equal to its focal length from the optical pattern, and thus collimates light from the optical pattern. The collimated light travels to a conventional camera. For example, the camera may be meters distant. The camera takes a photograph of the optical pattern at a time that the camera is not focused on the scene that it is imaging, but instead is focused at infinity. Because the light is collimated, however, a focused image is captured at the camera's focal plane. The binary data in the pattern may include information regarding the object to which the optical pattern is affixed and information from which the camera's pose may be calculated.

Abstract translation: 在本发明的说明性实现中，将二进制数据编码的光学图案印刷在透明体上。 例如，模式可以包括数据矩阵代码。 将小透镜从光学图案放置在与其焦距相等的距离处，从而准直来自光学图案的光。 准直光线传播到传统照相机。 例如，相机可能距离很远。 相机拍摄照相机在相机未对其成像的场景进行聚焦的时候拍摄光学图案，而是将其聚焦在无穷远处。 然而，因为光线是准直的，所以在相机焦平面上捕获聚焦图像。 图案中的二进制数据可以包括关于附加了光学图案的对象的信息和可以从其计算相机姿态的信息。

公开(公告)号：US20100259670A1

公开(公告)日：2010-10-14

申请号：US12758230

申请日：2010-04-12

Applicant: Ankit Mohan ,  Douglas Lanman ,  Shinsaku Hiura ,  Ramesh Raskar

Inventor： Ankit Mohan ,  Douglas Lanman ,  Shinsaku Hiura ,  Ramesh Raskar

IPC: H04N5/232

CPC classification number: H04N5/232 ,  H04N5/23248 ,  H04N5/23287 ,  H04N5/262

Abstract: In exemplary implements of this invention, a lens and sensor of a camera are intentionally destabilized (i.e., shifted relative to the scene being imaged) in order to create defocus effects. That is, actuators in a camera move a lens and a sensor, relative to the scene being imaged, while the camera takes a photograph. This motion simulates a larger aperture size (shallower depth of field). Thus, by translating a lens and a sensor while taking a photo, a camera with a small aperture (such as a cell phone or small point and shoot camera) may simulate the shallow DOF that can be achieved with a professional SLR camera. This invention may be implemented in such a way that programmable defocus effects may be achieved. Also, approximately depth-invariant defocus blur size may be achieved over a range of depths, in some embodiments of this invention.

Abstract translation: 在本发明的示例性实施例中，相机的透镜和传感器有意地不稳定（即，相对于被成像的场景移动），以便产生散焦效果。 也就是说，相机拍摄照片时，照相机中的致动器相对于被成像的场景移动透镜和传感器。 该运动模拟较大的孔径尺寸（较浅的景深）。 因此，通过在拍摄照片时平移透镜和传感器，具有小光圈的相机（例如手机或小点和拍摄相机）可以模拟使用专业SLR相机可以实现的浅DOF。 本发明可以以可实现可编程散焦效果的方式实现。 此外，在本发明的一些实施例中，可以在深度范围内实现大致深度不变的散焦模糊尺寸。