-
公开(公告)号:US08401225B2
公开(公告)日:2013-03-19
申请号:US13017626
申请日:2011-01-31
申请人: Richard Newcombe , Shahram Izadi , Otmar Hilliges , David Kim , David Molyneaux , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
发明人: Richard Newcombe , Shahram Izadi , Otmar Hilliges , David Kim , David Molyneaux , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
IPC分类号: G06K9/00
CPC分类号: G06T7/215 , G06T7/194 , G06T2207/10028 , G06T2207/30244
摘要: Moving object segmentation using depth images is described. In an example, a moving object is segmented from the background of a depth image of a scene received from a mobile depth camera. A previous depth image of the scene is retrieved, and compared to the current depth image using an iterative closest point algorithm. The iterative closest point algorithm includes a determination of a set of points that correspond between the current depth image and the previous depth image. During the determination of the set of points, one or more outlying points are detected that do not correspond between the two depth images, and the image elements at these outlying points are labeled as belonging to the moving object. In examples, the iterative closest point algorithm is executed as part of an algorithm for tracking the mobile depth camera, and hence the segmentation does not add substantial additional computational complexity.
摘要翻译: 描述使用深度图像来移动物体分割。 在一个示例中,从从移动深度相机接收的场景的深度图像的背景中分割移动物体。 检索场景的先前深度图像,并使用迭代最近点算法与当前深度图像进行比较。 迭代最近点算法包括对当前深度图像和先前深度图像之间对应的一组点的确定。 在确定点集合期间,检测到一个或多个在两个深度图像之间不对应的离开点,并且将这些离散点处的图像元素标记为属于移动对象。 在示例中,迭代最近点算法作为用于跟踪移动深度相机的算法的一部分被执行,因此分割不会增加实质的额外的计算复杂度。
-
公开(公告)号:US08401242B2
公开(公告)日:2013-03-19
申请号:US13017587
申请日:2011-01-31
申请人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
发明人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
IPC分类号: G06K9/00
CPC分类号: A63F13/00 , A63F13/06 , A63F2300/1087 , A63F2300/69 , G06K9/00 , G06K9/00664 , G06T7/251 , G06T7/30 , G06T2207/10028 , G06T2207/10048 , G06T2207/30244
摘要: Real-time camera tracking using depth maps is described. In an embodiment depth map frames are captured by a mobile depth camera at over 20 frames per second and used to dynamically update in real-time a set of registration parameters which specify how the mobile depth camera has moved. In examples the real-time camera tracking output is used for computer game applications and robotics. In an example, an iterative closest point process is used with projective data association and a point-to-plane error metric in order to compute the updated registration parameters. In an example, a graphics processing unit (GPU) implementation is used to optimize the error metric in real-time. In some embodiments, a dense 3D model of the mobile camera environment is used.
摘要翻译: 描述使用深度图的实时相机跟踪。 在一个实施例中,深度图帧由移动深度相机以每秒20帧的速度捕获,并且用于实时动态地更新一组注册参数,这些参数指定移动深度相机已经移动。 在实例中,实时相机跟踪输出用于计算机游戏应用和机器人。 在一个例子中,迭代最近点处理与投影数据关联和点到平面误差度量一起使用,以便计算更新的注册参数。 在一个例子中,使用图形处理单元(GPU)实现来实时优化误差度量。 在一些实施例中,使用移动照相机环境的密集3D模型。
-
公开(公告)号:US20120194650A1
公开(公告)日:2012-08-02
申请号:US13017518
申请日:2011-01-31
申请人: Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Stephen Edward Hodges , David Alexander Butler , Andrew Fitzgibbon , Pushmeet Kohli
发明人: Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Stephen Edward Hodges , David Alexander Butler , Andrew Fitzgibbon , Pushmeet Kohli
IPC分类号: H04N13/02
CPC分类号: G06T7/55 , G01B9/08 , G01B11/22 , H04N5/23229 , H04N5/247 , H04N5/33 , H04N5/332 , H04N13/271
摘要: Systems and methods for reducing interference between multiple infra-red depth cameras are described. In an embodiment, the system comprises multiple infra-red sources, each of which projects a structured light pattern into the environment. A controller is used to control the sources in order to reduce the interference caused by overlapping light patterns. Various methods are described including: cycling between the different sources, where the cycle used may be fixed or may change dynamically based on the scene detected using the cameras; setting the wavelength of each source so that overlapping patterns are at different wavelengths; moving source-camera pairs in independent motion patterns; and adjusting the shape of the projected light patterns to minimize overlap. These methods may also be combined in any way. In another embodiment, the system comprises a single source and a mirror system is used to cast the projected structured light pattern around the environment.
摘要翻译: 描述了用于减少多个红外深度摄像机之间的干扰的系统和方法。 在一个实施例中,系统包括多个红外源,每个红外源将结构化的光图案投射到环境中。 控制器用于控制源,以减少由重叠的光图案引起的干扰。 描述了各种方法,包括:在不同的源之间循环,其中使用的周期可以是固定的,或者可以基于使用相机检测的场景动态地改变; 设置每个源的波长,使得重叠图案处于不同的波长; 以独立运动模式移动源摄像机对; 并调整投影光图案的形状以最小化重叠。 这些方法也可以以任何方式组合。 在另一个实施例中,系统包括单个源,并且使用镜子系统将投射的结构化光图案围绕环境投射。
-
公开(公告)号:US08570320B2
公开(公告)日:2013-10-29
申请号:US13017729
申请日:2011-01-31
申请人: Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
发明人: Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
CPC分类号: G06F3/011 , A63F13/65 , A63F2300/1087 , A63F2300/6009 , G06T7/20 , G06T17/00 , G06T2207/10016 , G06T2207/10028 , G06T2207/30244
摘要: Use of a 3D environment model in gameplay is described. In an embodiment, a mobile depth camera is used to capture a series of depth images as it is moved around and a dense 3D model of the environment is generated from this series of depth images. This dense 3D model is incorporated within an interactive application, such as a game. The mobile depth camera is then placed in a static position for an interactive phase, which in some examples is gameplay, and the system detects motion of a user within a part of the environment from a second series of depth images captured by the camera. This motion provides a user input to the interactive application, such as a game. In further embodiments, automatic recognition and identification of objects within the 3D model may be performed and these identified objects then change the way that the interactive application operates.
摘要翻译: 描述了在游戏中使用3D环境模型。 在一个实施例中,移动深度相机被用来捕获一系列深度图像,因为它被移动,并且从该系列深度图像生成环境的密集3D模型。 这种密集的3D模型被并入在诸如游戏的交互式应用中。 然后将移动深度相机放置在用于交互式相位的静态位置,在一些示例中为游戏画面,并且系统从相机拍摄的第二系列深度图像中检测环境部分内的用户的运动。 该运动向诸如游戏的交互式应用提供用户输入。 在另外的实施例中,可以执行3D模型内的对象的自动识别和识别,并且这些识别的对象然后改变交互式应用程序的操作方式。
-
公开(公告)号:US20120194644A1
公开(公告)日:2012-08-02
申请号:US13017474
申请日:2011-01-31
申请人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
发明人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
IPC分类号: H04N5/225
CPC分类号: G06T7/20 , G06T7/74 , G06T2207/10016 , G06T2207/10021 , G06T2207/10024 , G06T2207/10028 , G06T2207/30244
摘要: Mobile camera localization using depth maps is described for robotics, immersive gaming, augmented reality and other applications. In an embodiment a mobile depth camera is tracked in an environment at the same time as a 3D model of the environment is formed using the sensed depth data. In an embodiment, when camera tracking fails, this is detected and the camera is relocalized either by using previously gathered keyframes or in other ways. In an embodiment, loop closures are detected in which the mobile camera revisits a location, by comparing features of a current depth map with the 3D model in real time. In embodiments the detected loop closures are used to improve the consistency and accuracy of the 3D model of the environment.
摘要翻译: 使用深度图的移动摄像机定位被描述为机器人,沉浸式游戏,增强现实和其他应用。 在一个实施例中,移动深度相机在环境中跟踪,同时使用感测的深度数据形成环境的3D模型。 在一个实施例中,当相机跟踪失败时,检测到该相机并且通过使用先前收集的关键帧或以其它方式来重新定位相机。 在一个实施例中,通过将当前深度图与3D模型的特征实时比较,检测到环路闭合,其中移动摄像机重新访问位置。 在实施例中,检测到的环路闭合用于改善环境的3D模型的一致性和准确性。
-
公开(公告)号:US09247238B2
公开(公告)日:2016-01-26
申请号:US13017518
申请日:2011-01-31
申请人: Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Stephen Edward Hodges , David Alexander Butler , Andrew Fitzgibbon , Pushmeet Kohli
发明人: Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Stephen Edward Hodges , David Alexander Butler , Andrew Fitzgibbon , Pushmeet Kohli
CPC分类号: G06T7/55 , G01B9/08 , G01B11/22 , H04N5/23229 , H04N5/247 , H04N5/33 , H04N5/332 , H04N13/271
摘要: Systems and methods for reducing interference between multiple infra-red depth cameras are described. In an embodiment, the system comprises multiple infra-red sources, each of which projects a structured light pattern into the environment. A controller is used to control the sources in order to reduce the interference caused by overlapping light patterns. Various methods are described including: cycling between the different sources, where the cycle used may be fixed or may change dynamically based on the scene detected using the cameras; setting the wavelength of each source so that overlapping patterns are at different wavelengths; moving source-camera pairs in independent motion patterns; and adjusting the shape of the projected light patterns to minimize overlap. These methods may also be combined in any way. In another embodiment, the system comprises a single source and a mirror system is used to cast the projected structured light pattern around the environment.
摘要翻译: 描述了用于减少多个红外深度摄像机之间的干扰的系统和方法。 在一个实施例中,系统包括多个红外源,每个红外源将结构化的光图案投射到环境中。 控制器用于控制源,以减少由重叠的光图案引起的干扰。 描述了各种方法,包括:在不同的源之间循环,其中使用的周期可以是固定的,或者可以基于使用相机检测的场景动态地改变; 设置每个源的波长,使得重叠图案处于不同的波长; 以独立运动模式移动源摄像机对; 并调整投影光图案的形状以最小化重叠。 这些方法也可以以任何方式组合。 在另一个实施例中,系统包括单个源,并且使用镜子系统将投射的结构化光图案围绕环境投射。
-
公开(公告)号:US08711206B2
公开(公告)日:2014-04-29
申请号:US13017474
申请日:2011-01-31
申请人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
发明人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
CPC分类号: G06T7/20 , G06T7/74 , G06T2207/10016 , G06T2207/10021 , G06T2207/10024 , G06T2207/10028 , G06T2207/30244
摘要: Mobile camera localization using depth maps is described for robotics, immersive gaming, augmented reality and other applications. In an embodiment a mobile depth camera is tracked in an environment at the same time as a 3D model of the environment is formed using the sensed depth data. In an embodiment, when camera tracking fails, this is detected and the camera is relocalized either by using previously gathered keyframes or in other ways. In an embodiment, loop closures are detected in which the mobile camera revisits a location, by comparing features of a current depth map with the 3D model in real time. In embodiments the detected loop closures are used to improve the consistency and accuracy of the 3D model of the environment.
摘要翻译: 使用深度图的移动摄像机定位被描述为机器人,沉浸式游戏,增强现实和其他应用。 在一个实施例中,移动深度相机在环境中跟踪,同时使用感测的深度数据形成环境的3D模型。 在一个实施例中,当相机跟踪失败时,检测到该相机并且通过使用先前收集的关键帧或以其它方式来重新定位相机。 在一个实施例中,通过将当前深度图与3D模型的特征实时比较,检测到环路闭合,其中移动摄像机重新访问位置。 在实施例中,检测到的环路闭合用于改善环境的3D模型的一致性和准确性。
-
公开(公告)号:US20120196679A1
公开(公告)日:2012-08-02
申请号:US13017587
申请日:2011-01-31
申请人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
发明人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
CPC分类号: A63F13/00 , A63F13/06 , A63F2300/1087 , A63F2300/69 , G06K9/00 , G06K9/00664 , G06T7/251 , G06T7/30 , G06T2207/10028 , G06T2207/10048 , G06T2207/30244
摘要: Real-time camera tracking using depth maps is described. In an embodiment depth map frames are captured by a mobile depth camera at over 20 frames per second and used to dynamically update in real-time a set of registration parameters which specify how the mobile depth camera has moved. In examples the real-time camera tracking output is used for computer game applications and robotics. In an example, an iterative closest point process is used with projective data association and a point-to-plane error metric in order to compute the updated registration parameters. In an example, a graphics processing unit (GPU) implementation is used to optimize the error metric in real-time. In some embodiments, a dense 3D model of the mobile camera environment is used.
摘要翻译: 描述使用深度图的实时相机跟踪。 在一个实施例中,深度图帧由移动深度相机以每秒20帧的速度捕获,并且用于实时动态地更新一组注册参数,这些参数指定移动深度相机已经移动。 在实例中,实时相机跟踪输出用于计算机游戏应用和机器人。 在一个例子中,迭代最近点处理与投影数据关联和点到平面误差度量一起使用,以便计算更新的注册参数。 在一个例子中,使用图形处理单元(GPU)实现来实时优化误差度量。 在一些实施例中,使用移动照相机环境的密集3D模型。
-
公开(公告)号:US08587583B2
公开(公告)日:2013-11-19
申请号:US13017690
申请日:2011-01-31
申请人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Stephen Edward Hodges , David Alexander Butler , Andrew Fitzgibbon , Pushmeet Kohli
发明人: Richard Newcombe , Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Stephen Edward Hodges , David Alexander Butler , Andrew Fitzgibbon , Pushmeet Kohli
CPC分类号: G06T17/00 , G06T2200/08
摘要: Three-dimensional environment reconstruction is described. In an example, a 3D model of a real-world environment is generated in a 3D volume made up of voxels stored on a memory device. The model is built from data describing a camera location and orientation, and a depth image with pixels indicating a distance from the camera to a point in the environment. A separate execution thread is assigned to each voxel in a plane of the volume. Each thread uses the camera location and orientation to determine a corresponding depth image location for its associated voxel, determines a factor relating to the distance between the associated voxel and the point in the environment at the corresponding location, and updates a stored value at the associated voxel using the factor. Each thread iterates through an equivalent voxel in the remaining planes of the volume, repeating the process to update the stored value.
摘要翻译: 描述了三维环境重建。 在一个示例中,在由存储在存储器件上的体素组成的3D体积中生成真实世界环境的3D模型。 该模型是从描述相机位置和方向的数据构建的,以及具有指示从相机到环境中的点的距离的像素的深度图像。 单独的执行线程被分配给卷的平面中的每个体素。 每个线程使用摄像机位置和方向来确定其相关体素的相应深度图像位置,确定与相关体素和相应位置处的环境中的点之间的距离有关的因子,并更新相关联的体素的存储值 体素使用因子。 每个线程遍历卷的剩余平面中的等效体素,重复更新存储值的过程。
-
公开(公告)号:US20120194517A1
公开(公告)日:2012-08-02
申请号:US13017729
申请日:2011-01-31
申请人: Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
发明人: Shahram Izadi , David Molyneaux , Otmar Hilliges , David Kim , Jamie Daniel Joseph Shotton , Pushmeet Kohli , Andrew Fitzgibbon , Stephen Edward Hodges , David Alexander Butler
IPC分类号: G06T17/00
CPC分类号: G06F3/011 , A63F13/65 , A63F2300/1087 , A63F2300/6009 , G06T7/20 , G06T17/00 , G06T2207/10016 , G06T2207/10028 , G06T2207/30244
摘要: Use of a 3D environment model in gameplay is described. In an embodiment, a mobile depth camera is used to capture a series of depth images as it is moved around and a dense 3D model of the environment is generated from this series of depth images. This dense 3D model is incorporated within an interactive application, such as a game. The mobile depth camera is then placed in a static position for an interactive phase, which in some examples is gameplay, and the system detects motion of a user within a part of the environment from a second series of depth images captured by the camera. This motion provides a user input to the interactive application, such as a game. In further embodiments, automatic recognition and identification of objects within the 3D model may be performed and these identified objects then change the way that the interactive application operates.
摘要翻译: 描述了在游戏中使用3D环境模型。 在一个实施例中,移动深度相机被用来捕获一系列深度图像,因为它被移动,并且从该系列深度图像生成环境的密集3D模型。 这种密集的3D模型被并入在诸如游戏的交互式应用中。 然后将移动深度相机放置在用于交互式相位的静态位置,在一些示例中为游戏画面,并且系统从相机拍摄的第二系列深度图像中检测环境部分内的用户的运动。 该运动向诸如游戏的交互式应用提供用户输入。 在另外的实施例中,可以执行3D模型内的对象的自动识别和识别,并且这些识别的对象然后改变交互式应用程序的操作方式。
-
-
-
-
-
-
-
-
-