公开(公告)号：US09343020B2

公开(公告)日：2016-05-17

申请号：US14451666

申请日：2014-08-05

Applicant: Felix Heide ,  Gordon Wetzstein ,  James Gregson ,  Ramesh Raskar ,  Wolfgang Heidrich

Inventor： Felix Heide ,  Gordon Wetzstein ,  James Gregson ,  Ramesh Raskar ,  Wolfgang Heidrich

IPC: G09G3/36

CPC classification number: G09G3/36 ,  G09G2300/023 ,  G09G2340/0457

Abstract: In exemplary implementations of this invention, light from a backlight is transmitted through two stacked LCDs and then through a diffuser. The front side of the diffuser displays a time-varying sequence of 2D images. Processors execute an optimization algorithm to compute optimal pixel states in the first and second LCDs, respectively, such that for each respective image in the sequence, the optimal pixel states minimize, subject to one or more constraints, a difference between a target image and the respective image. The processors output signals to control actual pixel states in the LCDs, based on the computed optimal pixel states. The 2D images displayed by the diffuser have a higher spatial resolution than the native spatial resolution of the LCDs. Alternatively, the diffuser may be switched off, and the device may display either (a) 2D images with a higher dynamic range than the LCDs, or (b) an automultiscopic display.

Abstract translation: 在本发明的示例性实施方案中，来自背光的光通过两个堆叠的LCD传输，然后通过扩散器传输。 漫射器的前侧显示2D图像的时变序列。 处理器执行优化算法以分别计算第一和第二LCD中的最佳像素状态，使得对于序列中的每个相应图像，最佳像素状态在受到一个或多个约束的情况下最小化目标图像与 各自的图像。 处理器基于所计算的最佳像素状态输出信号以控制LCD中的实际像素状态。 由漫射器显示的2D图像具有比LCD的原始空间分辨率更高的空间分辨率。 或者，可以关闭扩散器，并且该装置可以显示（a）具有比LCD更高的动态范围的2D图像，或（b）自动显微镜显示器。

公开(公告)号：US09295388B2

公开(公告)日：2016-03-29

申请号：US14214950

申请日：2014-03-16

Applicant: Matthew Everett Lawson ,  Ramesh Raskar

Inventor： Matthew Everett Lawson ,  Ramesh Raskar

IPC: A61B3/14 ,  A61B3/00 ,  A61B3/12 ,  G06K9/00

CPC classification number: A61B3/14 ,  A61B3/0025 ,  A61B3/0091 ,  A61B3/12 ,  G06K9/00604

Abstract: In exemplary implementations, this invention comprises apparatus for retinal self-imaging. Visual stimuli help the user self-align his eye with a camera. Bi-ocular coupling induces the test eye to rotate into different positions. As the test eye rotates, a video is captured of different areas of the retina. Computational photography methods process this video into a mosaiced image of a large area of the retina. An LED is pressed against the skin near the eye, to provide indirect, diffuse illumination of the retina. The camera has a wide field of view, and can image part of the retina even when the eye is off-axis (when the eye's pupillary axis and camera's optical axis are not aligned). Alternately, the retina is illuminated directly through the pupil, and different parts of a large lens are used to image different parts of the retina. Alternately, a plenoptic camera is used for retinal imaging.

公开(公告)号：US09100562B2

公开(公告)日：2015-08-04

申请号：US12758230

申请日：2010-04-12

Applicant: Ankit Mohan ,  Douglas Lanman ,  Shinsaku Hiura ,  Ramesh Raskar

Inventor： Ankit Mohan ,  Douglas Lanman ,  Shinsaku Hiura ,  Ramesh Raskar

IPC: H04N5/232 ,  H04N5/262

CPC classification number: H04N5/232 ,  H04N5/23248 ,  H04N5/23287 ,  H04N5/262

Abstract: In exemplary implements of this invention, a lens and sensor of a camera are intentionally destabilized (i.e., shifted relative to the scene being imaged) in order to create defocus effects. That is, actuators in a camera move a lens and a sensor, relative to the scene being imaged, while the camera takes a photograph. This motion simulates a larger aperture size (shallower depth of field). Thus, by translating a lens and a sensor while taking a photo, a camera with a small aperture (such as a cell phone or small point and shoot camera) may simulate the shallow DOF that can be achieved with a professional SLR camera. This invention may be implemented in such a way that programmable defocus effects may be achieved. Also, approximately depth-invariant defocus blur size may be achieved over a range of depths, in some embodiments of this invention.

Abstract translation: 在本发明的示例性实施例中，相机的透镜和传感器有意地不稳定（即，相对于被成像的场景移动），以便产生散焦效果。 也就是说，相机拍摄照片时，照相机中的致动器相对于被成像的场景移动透镜和传感器。 该运动模拟较大的孔径尺寸（较浅的景深）。 因此，通过在拍摄照片时平移透镜和传感器，具有小光圈的相机（例如手机或小点和拍摄相机）可以模拟使用专业SLR相机可以实现的浅DOF。 本发明可以以可实现可编程散焦效果的方式实现。 此外，在本发明的一些实施例中，可以在深度范围内实现大致深度不变的散焦模糊尺寸。

公开(公告)号：US20140368728A1

公开(公告)日：2014-12-18

申请号：US14304566

申请日：2014-06-13

Applicant: Ramesh Raskar ,  Christopher Barsi

Inventor： Ramesh Raskar ,  Christopher Barsi

IPC: H04N5/235 ,  H04N5/243

CPC classification number: H04N5/243 ,  G01J11/00 ,  H01J31/50 ,  H04N5/2354 ,  H04N5/30

Abstract: In exemplary implementations of this invention, a light source illuminates a scene and a light sensor captures data about light that scatters from the scene. The light source emits multiple modulation frequencies, either in a temporal sequence or as a superposition of modulation frequencies. Reference signals that differ in phase are applied to respective subregions of each respective pixel. The number of subregions per pixel, and the number of reference signals per pixel, is preferably greater than four. One or more processors calculate a full cross-correlation function for each respective pixel, by fitting light intensity measurements to a curve, the light intensity measurements being taken, respectively, by respective subregions of the respective pixel. The light sensor comprises M subregions. A lenslet is placed over each subregion, so that each subregion images the entire scene. At least one temporal sequence of frames is taken, one frame per subregion.

Abstract translation: 在本发明的示例性实施方案中，光源照亮场景，光传感器捕获关于从场景散射的光的数据。 光源以时间序列或调制频率的叠加发射多个调制频率。 不同相位的参考信号被施加到每个相应像素的相应子区域。 每像素的子区域的数量和每像素的参考信号的数量优选大于4。 一个或多个处理器通过将光强度测量结合到曲线上，分别由相应像素的相应子区域分别进行光强测量，来计算每个相应像素的完全互相关函数。 光传感器包括M个子区域。 每个子区域都放置一个小透镜，以便每个子区域对整个场景进行成像。 采取帧的至少一个时间序列，每个子区域一个帧。

公开(公告)号：US20140367558A1

公开(公告)日：2014-12-18

申请号：US14304507

申请日：2014-06-13

Applicant: Ramesh Raskar ,  Christopher Barsi

Inventor： Ramesh Raskar ,  Christopher Barsi

IPC: G01J1/42

CPC classification number: H04N5/243 ,  G01J11/00 ,  H01J31/50 ,  H04N5/2354 ,  H04N5/30

Abstract: In exemplary implementations of this invention, a camera can capture multiple millions of frames per second, such that each frame is 2D image, rather than a streak. A light source in the camera emits ultrashort pulses of light to illuminate a scene. Scattered light from the scene returns to the camera. This incoming light strikes a photocathode, which emits electrons, which are detected by a set of phosphor blocks, which emit light, which is detected by a light sensor. Voltage is applied to plates to create an electric field that deflects the electrons. The voltage varies in a temporal “stepladder” pattern, deflecting the electrons by different amounts, such that the electrons hit different phosphor blocks at different times during the sequence. Each phosphor block (together with the light sensor) captures a separate frame in the sequence. A mask may be used to increase resolution.

Abstract translation: 在本发明的示例性实现中，相机可以每秒捕获数百万帧，使得每个帧是2D图像，而不是条纹。 相机中的光源会发出超短脉冲的光线照亮场景。 来自场景的散射光返回到相机。 该入射光照射发射电子的光电阴极，该光电子由一组由光传感器检测到的发光的磷光体块检测。 将电压施加到板上以产生使电子偏转的电场。 电压在时间“梯形图”模式中变化，使电子偏转不同的量，使得电子在序列期间的不同时间击中不同的荧光体块。 每个荧光体块（与光传感器一起）在序列中捕获单独的帧。 可以使用掩模来提高分辨率。

公开(公告)号：US20140340569A1

公开(公告)日：2014-11-20

申请号：US14280284

申请日：2014-05-16

Applicant: Ramesh Raskar ,  Achuta Kadambi ,  Ayush Bhandari ,  Christopher Barsi

Inventor： Ramesh Raskar ,  Achuta Kadambi ,  Ayush Bhandari ,  Christopher Barsi

IPC: H04N5/235

CPC classification number: G01S17/89 ,  G01S7/493

Abstract: In exemplary implementations of this invention, a multi-frequency ToF camera mitigates the effect of multi-path interference (MPI), and can calculate an accurate depth map despite MPI. A light source in the multi-frequency camera emits light in a temporal sequence of different frequencies. For example, the light source can emit a sequence of ten equidistant frequencies f=10 MHz, 20 MHz, 30 MHz, . . . , 100 MHz. At each frequency, a lock-in sensor within the ToF camera captures 4 frames. From these 4 frames, one or more processors compute, for each pixel in the sensor, a single complex number. The processors stack all of such complex quantities (one such complex number per pixel per frequency) and solve for the depth and intensity, using a spectral estimation technique.

Abstract translation: 在本发明的示例性实施方案中，多频ToF相机减轻了多路径干扰（MPI）的影响，并且可以计算精确的深度图，尽管MPI。 多频摄像机中的光源以不同频率的时间序列发光。 例如，光源可以发射十个等距离的频率f = 10MHz，20MHz，30MHz的序列。 。 。 ，100MHz。 在每个频率下，ToF摄像头内的锁定传感器捕获4帧。 从这4个帧中，一个或多个处理器为传感器中的每个像素计算单个复数。 处理器堆叠所有这样的复杂数量（每个频率每个像素一个这样的复数），并使用频谱估计技术来解决深度和强度。

公开(公告)号：US08493432B2

公开(公告)日：2013-07-23

申请号：US12825608

申请日：2010-06-29

Applicant: Yuichi Taguchi ,  Amit K. Agrawal ,  Ashok N. Veeraraghavan ,  Srikumar Ramalingam ,  Ramesh Raskar

Inventor： Yuichi Taguchi ,  Amit K. Agrawal ,  Ashok N. Veeraraghavan ,  Srikumar Ramalingam ,  Ramesh Raskar

IPC: H04N7/00

CPC classification number: G06T5/50 ,  G06T2200/21 ,  G06T2207/10052 ,  H04N5/23238

Abstract: A single camera acquires an input image of a scene as observed in an array of spheres, wherein pixels in the input image corresponding to each sphere form a sphere image. A set of virtual cameras are defined for each sphere on a line joining a center of the sphere and a center of projection of the camera, wherein each virtual camera has a different virtual viewpoint and an associated cone of rays, appearing as a circle of pixels on its virtual image plane. A projective texture mapping of each sphere image is applied to all of the virtual cameras on the virtual image plane to produce a virtual camera image comprising circle of pixels. Each virtual camera image for each sphere is then projected to a refocusing geometry using a refocus viewpoint to produce a wide-angle lightfield view, which are averaged to produce a refocused wide-angle image.

Abstract translation: 单个相机获取在球体阵列中观察到的场景的输入图像，其中与每个球体对应的输入图像中的像素形成球面图像。 在连接球体中心的线和摄像机的投影中心的线上为每个球体定义一组虚拟照相机，其中每个虚拟照相机具有不同的虚拟视点和相关联的光线锥，呈现为像素圆 在其虚拟图像平面上。 每个球体图像的投影纹理映射被应用于虚拟图像平面上的所有虚拟相机，以产生包括像素圆的虚拟相机图像。 然后使用重聚焦点将每个球体的每个虚拟照相机图像投影到重聚焦几何，以产生广角光场视图，其被平均以产生重新聚焦的广角图像。

公开(公告)号：US08366003B2

公开(公告)日：2013-02-05

申请号：US12838304

申请日：2010-07-16

Applicant: Ankit Mohan ,  Ramesh Raskar ,  Shinsaku Hiura ,  Quinn Smithwick ,  Grace Woo

Inventor： Ankit Mohan ,  Ramesh Raskar ,  Shinsaku Hiura ,  Quinn Smithwick ,  Grace Woo

IPC: G06K7/10

CPC classification number: G06K7/10722 ,  G06K7/10831

Abstract: In an illustrative implementation of this invention, an optical pattern that encodes binary data is printed on a transparency. For example, the pattern may comprise data matrix codes. A lenslet is placed at a distance equal to its focal length from the optical pattern, and thus collimates light from the optical pattern. The collimated light travels to a conventional camera. For example, the camera may be meters distant. The camera takes a photograph of the optical pattern at a time that the camera is not focused on the scene that it is imaging, but instead is focused at infinity. Because the light is collimated, however, a focused image is captured at the camera's focal plane. The binary data in the pattern may include information regarding the object to which the optical pattern is affixed and information from which the camera's pose may be calculated.

Abstract translation: 在本发明的说明性实现中，将二进制数据编码的光学图案印刷在透明体上。 例如，该模式可以包括数据矩阵代码。 将小透镜从光学图案放置在与其焦距相等的距离处，从而准直来自光学图案的光。 准直光线传播到传统照相机。 例如，相机可能距离很远。 相机拍摄照相机在相机未对其成像的场景进行聚焦的时候拍摄光学图案，而是将其聚焦在无穷远处。 然而，因为光线是准直的，所以在相机焦平面上捕获聚焦图像。 图案中的二进制数据可以包括关于附加了光学图案的对象的信息和可以从其计算相机姿态的信息。

公开(公告)号：US08229244B2

公开(公告)日：2012-07-24

申请号：US12414162

申请日：2009-03-30

Applicant: Amit Agrawal ,  Yi Xu ,  Ramesh Raskar

Inventor： Amit Agrawal ,  Yi Xu ,  Ramesh Raskar

IPC: G06K9/40 ,  H04N5/225

CPC classification number: G06T5/003 ,  G06T5/20 ,  G06T5/50 ,  G06T2207/10016 ,  G06T2207/10148 ,  G06T2207/20201

Abstract: Embodiments of the invention describe a method for reducing a blur in an image of a scene. First, we acquire a set of images of the scene, wherein each image in the set of images includes an object having a blur associated with a point spread function (PSF) forming a set of point spread functions (PSFs), wherein the set of PSFs is suitable for null-filling operation. Next, we invert jointly the set of images and the set of PSFs to produce an output image having a reduced blur.

Abstract translation: 本发明的实施例描述了一种用于减少场景图像中的模糊的方法。 首先，我们获取场景的一组图像，其中图像集合中的每个图像包括具有与形成一组点扩散函数（PSF）的点扩散函数（PSF）相关联的模糊的对象，其中， PSF适用于零填充操作。 接下来，我们共同颠倒图像集合和PSF集合以产生具有减小的模糊的输出图像。

公开(公告)号：US20120075423A1

公开(公告)日：2012-03-29

申请号：US12893863

申请日：2010-09-29

Applicant: Ahmed Kirmani ,  Ramesh Raskar ,  James Davis

Inventor： Ahmed Kirmani ,  Ramesh Raskar ,  James Davis

IPC: H04N15/00

CPC classification number: G01S7/481 ,  G01S7/4865 ,  G01S17/89 ,  H04N3/02 ,  H04N5/2256

Abstract: In illustrative implementations of this invention, multi-path analysis of transient illumination is used to reconstruct scene geometry, even of objects that are occluded from the camera. An ultrafast camera system is used. It comprises a photo-sensor (e.g., accurate in the picosecond range), a pulsed illumination source (e.g. a femtosecond laser) and a processor. The camera emits a very brief light pulse that strikes a surface and bounces. Depending on the path taken, part of the light may return to the camera after one, two, three or more bounces. The photo-sensor captures the returning light bounces in a three-dimensional time image I(x,y,t) for each pixel. The camera takes different angular samples from the same viewpoint, recording a five-dimensional STIR (Space Time Impulse Response). A processor analyzes onset information in the STIR to estimate pairwise distances between patches in the scene, and then employs isometric embedding to estimate patch coordinates.

Abstract translation: 在本发明的说明性实现中，瞬时照明的多路径分析被用于重建场景几何图形，甚至是从相机遮挡的对象。 使用超高速摄像系统。 它包括光传感器（例如，在皮秒范围内精确的），脉冲照明源（例如飞秒激光）和处理器。 相机发出非常短暂的光脉冲，撞击表面并弹跳。 根据所采取的路径，部分光线可能在一次，两次，三次或更多次弹跳后返回相机。 光传感器捕获每个像素的三维时间图像I（x，y，t）中的返回光反弹。 相机从相同的角度拍摄不同的角度样本，记录五维STIR（空间时间脉冲响应）。 处理器分析STIR中的发作信息以估计场景中的斑块之间的成对距离，然后使用等距嵌入来估计贴片坐标。