公开(公告)号：US06811264B2

公开(公告)日：2004-11-02

申请号：US10394314

申请日：2003-03-21

Applicant: Ramesh Raskar ,  Jeroen van Baar ,  Paul A. Beardsley

Inventor： Ramesh Raskar ,  Jeroen van Baar ,  Paul A. Beardsley

IPC: G03B2114

CPC classification number: G06T3/0081 ,  G03B37/04 ,  G06K9/2036 ,  G06T5/006 ,  H04N9/3147 ,  H04N9/3185 ,  H04N9/3194

Abstract: An apparatus for projecting an output image on a display surface includes a processing unit. The processing unit includes a microprocessor, a memory and an I/O interface connected by buses. A projector sub-system coupled to the processing unit is for display output images on the display surface. A camera sub-system couple to the processing unit is for acquiring input images reflecting a geometry of the display surface. The camera sub-system is in a fixed physical relationship to the projector sub-system. Internal sensors coupled to the processing unit are for determining an orientation of the projector sub-system and the camera sub-system with respect to the display surface.

Abstract translation: 用于在显示表面上投影输出图像的装置包括处理单元。 处理单元包括微处理器，存储器和通过总线连接的I / O接口。 耦合到处理单元的投影仪子系统用于在显示表面上显示输出图像。 耦合到处理单元的相机子系统用于获取反映显示表面几何形状的输入图像。 相机子系统与投影机子系统处于固定的物理关系。 耦合到处理单元的内部传感器用于确定投影仪子系统和相机子系统相对于显示表面的取向。

公开(公告)号：US06729733B1

公开(公告)日：2004-05-04

申请号：US10394684

申请日：2003-03-21

Applicant: Ramesh Raskar ,  Thomas H. Willwacher ,  Jeroen van Baar

Inventor： Ramesh Raskar ,  Thomas H. Willwacher ,  Jeroen van Baar

IPC: G03B2100

CPC classification number: H04N9/3147 ,  H04N5/74 ,  H04N9/12 ,  H04N9/3185 ,  H04N9/3194

Abstract: A method determines a largest rectangle on a display surface. A polygon L is drawn on a first depth plane having a depth z=1 in a depth buffer. A rectangle R is drawn with a predetermined aspect ratio on a second depth plane having a depth z=0. A center of projection is determined with a minimum depth z in a range [0,1] that maps the rectangle R into a largest rectangle S in the first depth plane so that the rectangle S remains completely inside the polygon L.

Abstract translation: 方法确定显示表面上的最大矩形。 多边形L在深度缓冲器中在具有深度z = 1的第一深度平面上绘制。 在深度为z = 0的第二深度平面上以预定的纵横比绘制矩形R. 确定投影中心的最小深度z在[...]的范围内，将矩形R映射到第一深度平面中的最大矩形S，使矩形S完全保留在多边形L的内部。

公开(公告)号：US06520647B2

公开(公告)日：2003-02-18

申请号：US09930532

申请日：2001-08-15

Applicant: Ramesh Raskar

Inventor： Ramesh Raskar

IPC: G03B2114

CPC classification number: H04N9/3185 ,  G03B21/00 ,  H04N9/3194

Abstract: A method corrects keystoning in a projector arbitrarily oriented with respect to a display surface. An elevation angle, a roll angle, and an azimuth angle of an optical axis of the projector are measured with respect to the display surface. A planar projective transformation matrix is determined from the elevation, roll, and azimuth angles. A source image to be projected by the projector is warped according to the planar projective transformation, and then projected onto the display surface.

公开(公告)号：US09482622B2

公开(公告)日：2016-11-01

申请号：US14677898

申请日：2015-04-02

Applicant: Munehiko Sato ,  Ramesh Raskar ,  Boxin Shi ,  Alex Olwal

Inventor： Munehiko Sato ,  Ramesh Raskar ,  Boxin Shi ,  Alex Olwal

IPC: G01N21/00 ,  G01N21/84 ,  G01N21/25 ,  G01N21/47 ,  G01N21/31

CPC classification number: G01N21/84 ,  G01N21/251 ,  G01N21/255 ,  G01N21/474 ,  G01N2021/3155 ,  G01N2021/3181 ,  G01N2021/479 ,  G01N2021/8444 ,  G01N2201/06113 ,  G01N2201/0612 ,  G01N2201/062 ,  G01N2201/0627 ,  G01N2201/12

Abstract: In illustrative implementations of this invention, light sources illuminate a surface with multi-spectral, multi-directional illumination that varies in direction, wavelength, coherence and collimation. One or more cameras capture images of the surface while the surface is illuminated under different lighting conditions. One or more computers take, as input, data indicative of or derived from the images, and determine a classification of the surface. Based on the computed classification, the computers output signals to control an I/O device, such that content displayed by the I/O device depends, at least in part, on the computed classification. In illustrative implementations, this invention accurately classifies a wide range of surfaces, including transparent surfaces, specular surfaces, and surfaces with few features.

公开(公告)号：US09451177B2

公开(公告)日：2016-09-20

申请号：US14304507

申请日：2014-06-13

Applicant: Ramesh Raskar ,  Christopher Barsi

Inventor： Ramesh Raskar ,  Christopher Barsi

IPC: G01J1/42 ,  H04N5/243 ,  H04N5/235 ,  H04N5/30 ,  H01J31/50 ,  G01J11/00

CPC classification number: H04N5/243 ,  G01J11/00 ,  H01J31/50 ,  H04N5/2354 ,  H04N5/30

Abstract: In exemplary implementations of this invention, a camera can capture multiple millions of frames per second, such that each frame is 2D image, rather than a streak. A light source in the camera emits ultrashort pulses of light to illuminate a scene. Scattered light from the scene returns to the camera. This incoming light strikes a photocathode, which emits electrons, which are detected by a set of phosphor blocks, which emit light, which is detected by a light sensor. Voltage is applied to plates to create an electric field that deflects the electrons. The voltage varies in a temporal “stepladder” pattern, deflecting the electrons by different amounts, such that the electrons hit different phosphor blocks at different times during the sequence. Each phosphor block (together with the light sensor) captures a separate frame in the sequence. A mask may be used to increase resolution.

Abstract translation: 在本发明的示例性实现中，相机可以每秒捕获数百万帧，使得每个帧是2D图像，而不是条纹。 相机中的光源会发出超短脉冲的光线照亮场景。 来自场景的散射光返回到相机。 该入射光照射发射电子的光电阴极，该光电子由一组由光传感器检测到的发光的磷光体块检测。 将电压施加到板上以产生使电子偏转的电场。 电压在时间“梯形图”模式中变化，使电子偏转不同的量，使得电子在序列期间的不同时间击中不同的荧光体块。 每个荧光体块（与光传感器一起）在序列中捕获单独的帧。 可以使用掩模来提高分辨率。

公开(公告)号：US09148649B2

公开(公告)日：2015-09-29

申请号：US13646703

申请日：2012-10-07

Applicant: Ramesh Raskar ,  Andreas Velten

Inventor： Ramesh Raskar ,  Andreas Velten

IPC: H04N13/02 ,  G01S17/89 ,  G01S7/48 ,  G01S7/484 ,  G01S7/487

CPC classification number: H04N13/254 ,  G01S7/4808 ,  G01S7/484 ,  G01S7/487 ,  G01S17/89 ,  H04N13/20

Abstract: In exemplary implementations of this invention, a 3D range camera “looks around a corner” to image a hidden object, using light that has bounced (reflected) off of a diffuse reflector. The camera can recover the 3D structure of the hidden object.

Abstract translation: 在本发明的示例性实施方案中，使用已经从漫反射器反弹（反射）的光，3D范围相机“环绕角落”来对隐藏物体成像。 相机可以恢复隐藏对象的3D结构。

公开(公告)号：US09146403B2

公开(公告)日：2015-09-29

申请号：US13309535

申请日：2011-12-01

Applicant: Douglas Lanman ,  Matthew Hirsch ,  Yun Hee Kim ,  Szymon Jakubczak ,  Ramesh Raskar

Inventor： Douglas Lanman ,  Matthew Hirsch ,  Yun Hee Kim ,  Szymon Jakubczak ,  Ramesh Raskar

IPC: G02B26/00 ,  G02F1/1335 ,  H04N13/04 ,  H04N15/00 ,  G02B27/22

CPC classification number: G02B27/2214 ,  G02B27/225 ,  G02B27/2264 ,  H04N13/307 ,  H04N13/315 ,  H04N13/32

Abstract: In exemplary implementations of this invention, two LCD screens display a multi-view 3D image that has both horizontal and vertical parallax, and that does not require a viewer to wear any special glasses. Each pixel in the LCDs can take on any value: the pixel can be opaque, transparent, or any shade between. For regions of the image that are adjacent to a step function (e.g., a depth discontinuity) and not adjacent to a sharp corner, the screens display local parallax barriers comprising many small slits. The barriers and the slits tend to be oriented perpendicular to the local angular gradient of the target light field. In some implementations, the display is optimized to seek to minimize the Euclidian distance between the desired light field and the actual light field that is produced. Weighted, non-negative matrix factorization (NMF) is used for this optimization.

Abstract translation: 在本发明的示例性实施方案中，两个LCD屏幕显示具有水平和垂直视差的多视图3D图像，并且不需要观看者佩戴任何特殊眼镜。 LCD中的每个像素可以承担任何值：像素可以是不透明的，透明的或任何阴影之间。 对于邻近阶梯函数（例如，深度不连续）并且不与锐角相邻的图像的区域，屏幕显示包括许多小缝隙的局部视差屏障。 障碍物和狭缝倾向于垂直于目标光场的局部角度梯度取向。 在一些实施方案中，显示器被优化以寻求使期望的光场和所产生的实际光场之间的欧几里德距离最小化。 加权非负矩阵因子分解（NMF）用于此优化。

公开(公告)号：US09106841B2

公开(公告)日：2015-08-11

申请号：US14304566

申请日：2014-06-13

Applicant: Ramesh Raskar ,  Christopher Barsi

Inventor： Ramesh Raskar ,  Christopher Barsi

IPC: H04N5/222 ,  H04N5/243 ,  H04N5/235 ,  H04N5/30 ,  H01J31/50 ,  G01J11/00

CPC classification number: H04N5/243 ,  G01J11/00 ,  H01J31/50 ,  H04N5/2354 ,  H04N5/30

Abstract: In exemplary implementations of this invention, a light source illuminates a scene and a light sensor captures data about light that scatters from the scene. The light source emits multiple modulation frequencies, either in a temporal sequence or as a superposition of modulation frequencies. Reference signals that differ in phase are applied to respective subregions of each respective pixel. The number of subregions per pixel, and the number of reference signals per pixel, is preferably greater than four. One or more processors calculate a full cross-correlation function for each respective pixel, by fitting light intensity measurements to a curve, the light intensity measurements being taken, respectively, by respective subregions of the respective pixel. The light sensor comprises M subregions. A lenslet is placed over each subregion, so that each subregion images the entire scene. At least one temporal sequence of frames is taken, one frame per subregion.

Abstract translation: 在本发明的示例性实施方案中，光源照亮场景，光传感器捕获关于从场景散射的光的数据。 光源以时间序列或调制频率的叠加发射多个调制频率。 不同相位的参考信号被施加到每个相应像素的相应子区域。 每像素的子区域的数量和每像素的参考信号的数量优选大于4。 一个或多个处理器通过将光强度测量结合到曲线上，分别由相应像素的相应子区域分别进行光强测量，来计算每个相应像素的完全互相关函数。 光传感器包括M个子区域。 每个子区域都放置一个小透镜，以便每个子区域对整个场景进行成像。 采取帧的至少一个时间序列，每个子区域一个帧。

公开(公告)号：US09081262B2

公开(公告)日：2015-07-14

申请号：US13646719

申请日：2012-10-07

Applicant: Ramesh Raskar ,  Andreas Velten

Inventor： Ramesh Raskar ,  Andreas Velten

IPC: G03B17/17 ,  G03B15/00 ,  G03B39/00 ,  H04N5/225

CPC classification number: G03B17/17 ,  G03B15/00 ,  G03B39/005 ,  H04N5/225 ,  H04N5/2251

Abstract: In exemplary implementations of this invention, a set of two scanning mirrors scans the one dimensional field of view of a streak camera across a scene. The mirrors are continuously moving while the camera takes streak images. Alternately, the mirrors may only between image captures. An illumination source or other captured event is synchronized with the camera so that for every streak image the scene looks different. The scanning assures that different parts of the scene are captured.

Abstract translation: 在本发明的示例性实施方案中，一组两个扫描反射镜在场景上扫描条纹相机的一维视场。 镜头连续移动，而相机拍摄条纹图像。 或者，镜子只能在图像捕捉之间。 照明源或其他捕获的事件与相机同步，使得对于每个条纹图像，场景看起来不同。 扫描确保捕获场景的不同部分。

公开(公告)号：US08749619B2

公开(公告)日：2014-06-10

申请号：US12893863

申请日：2010-09-29

Applicant: Ahmed Kirmani ,  Ramesh Raskar ,  James Davis

Inventor： Ahmed Kirmani ,  Ramesh Raskar ,  James Davis

IPC: H04N15/00 ,  G01S7/481 ,  G01S7/486 ,  G01S17/89 ,  H04N3/02 ,  H04N5/225

CPC classification number: G01S7/481 ,  G01S7/4865 ,  G01S17/89 ,  H04N3/02 ,  H04N5/2256

Abstract: In illustrative implementations of this invention, multi-path analysis of transient illumination is used to reconstruct scene geometry, even of objects that are occluded from the camera. An ultrafast camera system is used. It comprises a photo-sensor (e.g., accurate in the picosecond range), a pulsed illumination source (e.g. a femtosecond laser) and a processor. The camera emits a very brief light pulse that strikes a surface and bounces. Depending on the path taken, part of the light may return to the camera after one, two, three or more bounces. The photo-sensor captures the returning light bounces in a three-dimensional time image I(x,y,t) for each pixel. The camera takes different angular samples from the same viewpoint, recording a five-dimensional STIR (Space Time Impulse Response). A processor analyzes onset information in the STIR to estimate pairwise distances between patches in the scene, and then employs isometric embedding to estimate patch coordinates.

Abstract translation: 在本发明的说明性实现中，瞬时照明的多路径分析被用于重建场景几何图形，甚至是从相机遮挡的对象。 使用超高速摄像系统。 它包括光传感器（例如，在皮秒范围内精确的），脉冲照明源（例如飞秒激光）和处理器。 相机发出非常短暂的光脉冲，撞击表面并弹跳。 根据所采取的路径，部分光线可能在一次，两次，三次或更多次弹跳后返回相机。 光传感器捕获每个像素的三维时间图像I（x，y，t）中的返回光反弹。 相机从相同的角度拍摄不同的角度样本，记录五维STIR（空间时间脉冲响应）。 处理器分析STIR中的发作信息以估计场景中的斑块之间的成对距离，然后使用等距嵌入来估计贴片坐标。