Methods and apparatus for virtual sensor array

    公开(公告)号:US09897699B2

    公开(公告)日:2018-02-20

    申请号:US14795113

    申请日:2015-07-09

    摘要: A time-of-flight camera images an object around a corner or through a diffuser. In the case of imaging around a corner, light from a hidden target object reflects off a diffuse surface and travels to the camera. Points on the diffuse surface function as a virtual sensors. In the case of imaging through a diffuser, light from the target object is transmitted through a diffusive media and travels to the camera. Points on a surface of the diffuse media that is visible to the camera function as virtual sensors. In both cases, a computer represents phase and intensity measurements taken by the camera as a system of linear equations and solves a linear inverse problem to (i) recover an image of the target object; or (ii) to compute a 3D position for each point in a set of points on an exterior surface of the target object.

    Methods and apparatus for multi-frequency camera
    2.
    发明申请
    Methods and apparatus for multi-frequency camera 有权
    多频摄像机的方法和装置

    公开(公告)号:US20140340569A1

    公开(公告)日:2014-11-20

    申请号:US14280284

    申请日:2014-05-16

    IPC分类号: H04N5/235

    CPC分类号: G01S17/89 G01S7/493

    摘要: In exemplary implementations of this invention, a multi-frequency ToF camera mitigates the effect of multi-path interference (MPI), and can calculate an accurate depth map despite MPI. A light source in the multi-frequency camera emits light in a temporal sequence of different frequencies. For example, the light source can emit a sequence of ten equidistant frequencies f=10 MHz, 20 MHz, 30 MHz, . . . , 100 MHz. At each frequency, a lock-in sensor within the ToF camera captures 4 frames. From these 4 frames, one or more processors compute, for each pixel in the sensor, a single complex number. The processors stack all of such complex quantities (one such complex number per pixel per frequency) and solve for the depth and intensity, using a spectral estimation technique.

    摘要翻译: 在本发明的示例性实施方案中,多频ToF相机减轻了多路径干扰(MPI)的影响,并且可以计算精确的深度图,尽管MPI。 多频摄像机中的光源以不同频率的时间序列发光。 例如,光源可以发射十个等距离的频率f = 10MHz,20MHz,30MHz的序列。 。 。 ,100MHz。 在每个频率下,ToF摄像头内的锁定传感器捕获4帧。 从这4个帧中,一个或多个处理器为传感器中的每个像素计算单个复数。 处理器堆叠所有这样的复杂数量(每个频率每个像素一个这样的复数),并使用频谱估计技术来解决深度和强度。

    Methods and apparatus for multi-frequency camera
    4.
    发明授权
    Methods and apparatus for multi-frequency camera 有权
    多频摄像机的方法和装置

    公开(公告)号:US09405008B2

    公开(公告)日:2016-08-02

    申请号:US14280284

    申请日:2014-05-16

    CPC分类号: G01S17/89 G01S7/493

    摘要: In exemplary implementations of this invention, a multi-frequency ToF camera mitigates the effect of multi-path interference (MPI), and can calculate an accurate depth map despite MPI. A light source in the multi-frequency camera emits light in a temporal sequence of different frequencies. For example, the light source can emit a sequence of ten equidistant frequencies f=10 MHz, 20 MHz, 30 MHz, . . . , 100 MHz. At each frequency, a lock-in sensor within the ToF camera captures 4 frames. From these 4 frames, one or more processors compute, for each pixel in the sensor, a single complex number. The processors stack all of such complex quantities (one such complex number per pixel per frequency) and solve for the depth and intensity, using a spectral estimation technique.

    摘要翻译: 在本发明的示例性实施方案中,多频ToF相机减轻了多路径干扰(MPI)的影响,并且可以计算精确的深度图,尽管MPI。 多频摄像机中的光源以不同频率的时间序列发光。 例如,光源可以发射十个等距离的频率f = 10MHz,20MHz,30MHz的序列。 。 。 ,100MHz。 在每个频率下,ToF摄像头内的锁定传感器捕获4帧。 从这4个帧中,一个或多个处理器为传感器中的每个像素计算单个复数。 处理器堆叠所有这样的复杂数量(每个频率每个像素一个这样的复数),并使用频谱估计技术来解决深度和强度。

    Methods and Apparatus for Coded Time-of-Flight Camera
    5.
    发明申请
    Methods and Apparatus for Coded Time-of-Flight Camera 有权
    编码飞行时间相机的方法和装置

    公开(公告)号:US20150120241A1

    公开(公告)日:2015-04-30

    申请号:US14523708

    申请日:2014-10-24

    IPC分类号: G01S17/89

    摘要: In illustrative implementations, a time-of-flight camera robustly measures scene depths, despite multipath interference. The camera emits amplitude modulated light. An FPGA sends at least two electrical signals, the first being to control modulation of radiant power of a light source and the second being a reference signal to control modulation of pixel gain in a light sensor. These signals are identical, except for time delays. These signals comprise binary codes that are m-sequences or other broadband codes. The correlation waveform is not sinusoidal. During measurements, only one fundamental modulation frequency is used. One or more computer processors solve a linear system by deconvolution, in order to recover an environmental function. Sparse deconvolution is used if the scene has only a few objects at a finite depth. Another algorithm, such as Wiener deconvolution, is used is the scene has global illumination or a scattering media.

    摘要翻译: 在说明性实现中,尽管多路径干扰,飞行时间相机鲁棒地测量场景深度。 相机发射调幅光。 FPGA发送至少两个电信号,首先是控制光源的辐射功率的调制,第二个是参考信号,以控制光传感器中像素增益的调制。 这些信号是相同的,除了时间延迟。 这些信号包括作为m序列或其他宽带码的二进制码。 相关波形不是正弦波。 在测量期间,仅使用一个基本调制频率。 一个或多个计算机处理器通过去卷积解决线性系统,以便恢复环境功能。 如果场景在有限深度只有几个对象,则使用稀疏去卷积。 使用另一种算法,如Wiener去卷积,场景有全局照明或散射介质。

    Methods and apparatus for demultiplexing illumination
    6.
    发明授权
    Methods and apparatus for demultiplexing illumination 有权
    用于解复用照明的方法和装置

    公开(公告)号:US09451141B2

    公开(公告)日:2016-09-20

    申请号:US14690159

    申请日:2015-04-17

    摘要: In illustrative implementations of this invention, an imaging system includes multiple light sources that illuminate a scene, and also includes a lock-in time of flight camera. While the scene is illuminated by these light sources, each of the light sources is amplitude-modulated by a different modulation pattern, and a reference signal is applied to the lock-in time-of-flight camera. The modulation patterns and the reference signal are carefully chosen such that the imaging system is able to disentangle, in real time, the respective contributions of the different light sources, and to compute, in real-time, depth of the scene. In some cases, the modulation signals for the light sources are orthogonal to each other and the reference signal is broadband. In some cases, the modulation codes for the light sources and the reference code are optimal codes that are determined by an optimization algorithm.

    摘要翻译: 在本发明的说明性实现中,成像系统包括照亮场景的多个光源,并且还包括飞行时间相机的锁定时间。 当这些光源照亮场景时,每个光源被不同的调制模式进行幅度调制,并且将参考信号应用于锁定飞行时间相机。 仔细选择调制图案和参考信号,使得成像系统能够实时地解析不同光源的相应贡献,并且实时地计算场景的深度。 在某些情况下,光源的调制信号彼此正交,参考信号是宽带的。 在某些情况下,光源的调制码和参考码是由优化算法确定的最佳码。

    Methods and Apparatus for Virtual Sensor Array
    7.
    发明申请
    Methods and Apparatus for Virtual Sensor Array 有权
    虚拟传感器阵列的方法和装置

    公开(公告)号:US20160014393A1

    公开(公告)日:2016-01-14

    申请号:US14795113

    申请日:2015-07-09

    IPC分类号: H04N13/02 G01S17/89

    摘要: A time-of-flight camera images an object around a corner or through a diffuser. In the case of imaging around a corner, light from a hidden target object reflects off a diffuse surface and travels to the camera. Points on the diffuse surface function as a virtual sensors. In the case of imaging through a diffuser, light from the target object is transmitted through a diffusive media and travels to the camera. Points on a surface of the diffuse media that is visible to the camera function as virtual sensors. In both cases, a computer represents phase and intensity measurements taken by the camera as a system of linear equations and solves a linear inverse problem to (i) recover an image of the target object; or (ii) to compute a 3D position for each point in a set of points on an exterior surface of the target object.

    摘要翻译: 飞行时间相机在拐角处或通过扩散器拍摄物体。 在围绕角落成像的情况下,来自隐藏的目标物体的光从漫射表面反射并行进到照相机。 漫反射面上的点作为虚拟传感器。 在通过漫射器进行成像的情况下,来自目标物体的光通过漫射介质传播并传播到照相机。 相机可见的漫射介质表面上的点作为虚拟传感器。 在这两种情况下,计算机表示作为线性方程组的相机拍摄的相位和强度测量值,并解决了线性反问题(i)恢复目标对象的图像; 或(ii)计算目标对象的外表面上的一组点中的每个点的3D位置。

    Methods and Apparatus for Demultiplexing Illumination
    8.
    发明申请
    Methods and Apparatus for Demultiplexing Illumination 有权
    用于解复用照明的方法和装置

    公开(公告)号:US20150304534A1

    公开(公告)日:2015-10-22

    申请号:US14690159

    申请日:2015-04-17

    IPC分类号: H04N5/225 H04N5/232

    摘要: In illustrative implementations of this invention, an imaging system includes multiple light sources that illuminate a scene, and also includes a lock-in time of flight camera. While the scene is illuminated by these light sources, each of the light sources is amplitude-modulated by a different modulation pattern, and a reference signal is applied to the lock-in time-of-flight camera. The modulation patterns and the reference signal are carefully chosen such that the imaging system is able to disentangle, in real time, the respective contributions of the different light sources, and to compute, in real-time, depth of the scene. In some cases, the modulation signals for the light sources are orthogonal to each other and the reference signal is broadband. In some cases, the modulation codes for the light sources and the reference code are optimal codes that are determined by an optimization algorithm.

    摘要翻译: 在本发明的说明性实现中,成像系统包括照亮场景的多个光源,并且还包括飞行时间相机的锁定时间。 当这些光源照亮场景时,每个光源被不同的调制模式进行幅度调制,并且将参考信号应用于锁定飞行时间相机。 仔细选择调制图案和参考信号,使得成像系统能够实时地解析不同光源的相应贡献,并且实时地计算场景的深度。 在某些情况下,光源的调制信号彼此正交,参考信号是宽带的。 在某些情况下,光源的调制码和参考码是由优化算法确定的最佳码。