公开(公告)号：US20150009119A1

公开(公告)日：2015-01-08

申请号：US13987323

申请日：2013-01-29

申请人： Imimtek, Inc.

发明人： Tony Zuccarino ,  Abbas Rafii

IPC分类号： G06F3/01

CPC分类号： G06F3/017 ,  G06F3/0304

摘要： Systems and method are disclosed for enabling a user to interact with gestures in a natural way with image(s) displayed on the surface of an integrated monitor whose display contents are governed by an appliance, perhaps a PC, smart phone or tablet. Some embodiments include the display as well as the appliance, in a single package such as all-in-one computers. User interaction includes gestures that may occur within a three-dimensional hover zone spaced apart from the display surface.

摘要翻译： 公开了系统和方法，以使用户能够以自然的方式与显示内容由设备（可能是PC，智能电话或平板电脑）控制的集成监视器的表面上的图像进行交互。 一些实施例在诸如一体机的单个包装中包括显示器以及设备。 用户交互包括可能在与显示表面间隔开的三维悬停区域内发生的手势。

公开(公告)号：US20140173440A1

公开(公告)日：2014-06-19

申请号：US13899537

申请日：2013-05-21

申请人： IMIMTEK, INC.

发明人： Carlo Dal Mutto ,  Abbas Rafii

IPC分类号： G06F3/01

CPC分类号： G06F3/017 ,  G06F3/0304 ,  G06F3/167

摘要： Systems and methods for natural interaction with graphical user interfaces using gestural and vocal input in accordance with embodiments of the invention are disclosed. In one embodiment, a method for interpreting a command sequence that includes a gesture and a voice cue to issue an application command includes receiving image data, receiving an audio signal, selecting an application command from a command dictionary based upon a gesture identified using the image data, a voice cue identified using the audio signal, and metadata describing combinations of a gesture and a voice cue that form a command sequence corresponding to an application command, retrieving a list of processes running on an operating system, selecting at least one process based upon the selected application command and the metadata, where the metadata also includes information identifying at least one process targeted by the application command, and issuing an application command to the selected process.

摘要翻译： 公开了根据本发明的实施例的使用手势和声音输入的与图形用户界面自然相互作用的系统和方法。 在一个实施例中，用于解释包括手势和语音提示以发出应用命令的命令序列的方法包括接收图像数据，接收音频信号，基于使用图像识别的手势从命令字典中选择应用命令 数据，使用音频信号识别的语音提示，以及描述形成与应用命令相对应的命令序列的手势和语音提示的组合的元数据，检索在操作系统上运行的进程的列表，选择至少一个基于进程 在所选择的应用程序命令和元数据上，元数据还包括标识应用程序命令所针对的至少一个进程的信息，并向所选择的进程发出应用程序命令。

公开(公告)号：US08655021B2

公开(公告)日：2014-02-18

申请号：US13942662

申请日：2013-07-15

申请人： Imimtek, Inc.

发明人： Carlo Dal Mutto ,  Britta Hummel ,  Michele Stoppa ,  Giridhar Murali ,  Teresa Mao ,  Giuliano Pasqualotto

IPC分类号： G06K9/00 ,  G06K9/32

CPC分类号： G06K9/00355 ,  G06F21/32 ,  G06K9/00375

摘要： Systems and methods for tracking human hands by performing parts based template matching using images captured from multiple viewpoints are described. One embodiment includes a processor, a reference camera, an alternate view camera, and memory containing: a hand tracking application; and a plurality of edge feature templates that are rotated and scaled versions of a finger template that includes an edge features template. In addition, the hand tracking application configures the processor to: detect at least one candidate finger in a reference frame, where each candidate finger is a grouping of pixels identified by searching the reference frame for a grouping of pixels that have image gradient orientations that match one of the plurality of edge feature templates; and verify the correct detection of a candidate finger in the reference frame by locating a grouping of pixels in an alternate view frame that correspond to the candidate finger.

摘要翻译： 描述通过使用从多个视点捕获的图像执行基于零件的模板匹配来跟踪人手的系统和方法。 一个实施例包括处理器，参考相机，备用视图相机和存储器，其包含：手跟踪应用; 以及包括边缘特征模板的手指模板的旋转和缩放版本的多个边缘特征模板。 另外，手跟踪应用将处理器配置为：在参考帧中检测至少一个候选手指，其中每个候选手指是通过搜索参考帧而识别的像素分组，所述像素分组具有匹配的图像梯度取向 多个边缘特征模板之一; 并且通过将对应于候选手指的替代视图帧中的像素分组进行定位来验证参考帧中候选手指的正确检测。

公开(公告)号：US20140298273A1

公开(公告)日：2014-10-02

申请号：US13965157

申请日：2013-08-12

申请人： Imimtek, Inc.

发明人： Keith Blackstone ,  Eric Webb ,  David Girdwood ,  Carlo Dal Mutto

IPC分类号： G06F3/01

CPC分类号： G06F3/017 ,  G06F3/011 ,  G06F3/0304 ,  G06F2203/04806

摘要： Systems and methods in accordance with embodiments of the invention implement three-dimensional (3D) gesture based graphical user interfaces (GUI) using gesture reactive interface objects. One embodiment includes using a computing device to render an initial user interface comprising a set of interface objects, detect a targeting 3D gesture in captured image data that identifies a targeted interface object within the user interface, change the rendering of at least the targeted interface object within the user interface in response to the targeting 3D gesture that targets the interface object, detect an interaction 3D gesture in additional captured image data that identifies a specific interaction with a targeted interface object, modify the user interface in response to the interaction with the targeted interface object identified by the interaction 3D gesture, and render the modified user interface.

摘要翻译： 根据本发明实施例的系统和方法使用手势反应式接口对象实现基于三维（3D）手势的图形用户界面（GUI）。 一个实施例包括使用计算设备来呈现包括一组接口对象的初始用户界面，在捕获的图像数据中检测目标3D手势，所述捕获图像数据标识用户界面内的目标接口对象，改变至少目标接口对象的呈现 响应于针对接口对象的目标3D手势，在用户界面内检测识别与目标接口对象的特定交互的附加捕获图像数据中的交互3D手势，响应于与目标对象的交互来修改用户界面 通过交互3D手势识别的界面对象，并呈现修改后的用户界面。

公开(公告)号：US08723789B1

公开(公告)日：2014-05-13

申请号：US13385134

申请日：2012-02-03

申请人： Abbas Rafii

发明人： Abbas Rafii

IPC分类号： G06F3/033

CPC分类号： G06F3/017 ,  G06F3/011

摘要： User interaction with a display is detected using at least two cameras whose intersecting FOVs define a three-dimensional hover zone within which user interactions can be imaged. Each camera substantially simultaneously acquires from its vantage point two-dimensional images of the user within the hover zone. Separately and collectively the image data is analyzed to identify therein a relatively few landmarks definable on the user. A substantially unambiguous correspondence is established between the same landmark on each acquired image, and as to those landmarks a three-dimensional reconstruction is made in a common coordinate system. This landmark identification and position information can be converted into a command causing the display to respond appropriately to a gesture made by the user. Advantageously size of the hover zone can far exceed size of the display, making the invention usable with smart phones as well as large size entertainment TVs.

摘要翻译： 使用至少两个摄像机检测与显示器的用户交互，其相交的FOV定义了可以在其中映像用户交互的三维悬停区域。 每个相机基本上同时从悬停区域内的用户的有利位置获取二维图像。 单独地和共同地分析图像数据以在其中识别用户可定义的相对少的地标。 在每个获取的图像上的相同标记之间建立基本上明确的对应关系，并且对于那些地标，在公共坐标系中进行三维重建。 该地标识别和位置信息可以被转换成导致显示器对用户做出的手势的适当响应的命令。 有利地，悬停区域的尺寸可以远远超过显示器的尺寸，使得本发明可用于智能电话以及大尺寸娱乐电视机。

公开(公告)号：US08686943B1

公开(公告)日：2014-04-01

申请号：US13506743

申请日：2012-05-14

申请人： Abbas Rafii

发明人： Abbas Rafii

IPC分类号： G06F3/033

CPC分类号： G06F3/042 ,  G06F3/011 ,  G06F3/0304 ,  G06F2203/04104 ,  G06F2203/04108 ,  G06K9/00214 ,  G06K9/00389 ,  G06K9/4671

摘要： User interaction with a display is detected substantially simultaneously using at least two cameras whose intersecting FOVs define a three-dimensional hover zone within which user interactions can be imaged. Separately and collectively image data is analyzed to identify a relatively few user landmarks. A substantially unambiguous correspondence is established between the same landmark on each acquired image, and a three-dimensional reconstruction is made in a common coordinate system. Preferably cameras are modeled to have characteristics of pinhole cameras, enabling rectified epipolar geometric analysis to facilitate more rapid disambiguation among potential landmark points. Consequently processing overhead is substantially reduced, as are latency times. Landmark identification and position information is convertible into a command causing the display to respond appropriately to a user gesture. Advantageously size of the hover zone can far exceed size of the display, making the invention usable with smart phones as well as large size entertainment TVs.

摘要翻译： 基本上同时使用至少两个摄像机来检测与显示器的用户交互，其中相机的FOV定义可以在其中映像用户交互的三维悬停区域。 分析和统一图像数据被分析以识别相对较少的用户地标。 在每个获取的图像上的相同标记之间建立基本上明确的对应关系，并且在公共坐标系中进行三维重建。 优选的是，相机被建模为具有针孔相机的特征，使矫正的对极几何分析能够有助于在潜在的地标点之间更快地消除歧义。 因此，延迟时间也大大降低了处理开销。 地标识别和位置信息可转换成导致显示器对用户手势适当响应的命令。 有利地，悬停区域的尺寸可以远远超过显示器的尺寸，使得本发明可用于智能电话以及大尺寸娱乐电视机。

公开(公告)号：US20140211991A1

公开(公告)日：2014-07-31

申请号：US13900015

申请日：2013-05-22

申请人： IMIMTEK, INC.

发明人： Michele Stoppa ,  Britta Hummel ,  Carlo Dal Mutto ,  Giuliano Pasqualotto

IPC分类号： G06K9/00 ,  G06T7/20

CPC分类号： G06K9/00355 ,  G06T7/251 ,  G06T2207/30196

摘要： Systems and methods for initializing motion tracking of human hands are disclosed. One embodiment includes a processor; a reference camera; and memory containing: a hand tracking application; and a plurality of edge feature templates that are rotated and scaled versions of a base template. The hand tracking application configures the processor to: determine whether any pixels in a frame of video are part of a human hand, where a part of a human hand is identified by searching the frame of video data for a grouping of pixels that have image gradient orientations that match the edge features of one of the plurality of edge feature templates; track the motion of the part of the human hand visible in a sequence of frames of video; confirm that the tracked motion corresponds to an initialization gesture; and commence tracking the human hand as part of a gesture based interactive session.

摘要翻译： 公开了用于初始化人手的运动跟踪的系统和方法。 一个实施例包括处理器; 参考相机; 和记忆包含：手跟踪应用程序; 以及基本模板的旋转和缩放版本的多个边缘特征模板。 手跟踪应用将处理器配置为：确定视频帧中的任何像素是否是人的手的一部分，其中通过搜索具有图像梯度的像素组的视频数据的帧来识别人的手的一部分 与所述多个边缘特征模板之一的边缘特征相匹配的取向; 跟踪视频的一系列帧中可见的人类手部的运动; 确认跟踪运动对应于初始化手势; 并开始跟踪人手作为基于手势的交互式会话的一部分。

公开(公告)号：US20130343606A1

公开(公告)日：2013-12-26

申请号：US13899536

申请日：2013-05-21

申请人： Imimtek, Inc.

发明人： Carlo Dal Mutto ,  Britta Hummel ,  Michele Stoppa ,  Giulian Pasqualotto ,  Teresa Mao ,  Murali Giridhar

IPC分类号： G06K9/00

CPC分类号： G06K9/00355 ,  G06F21/32 ,  G06K9/00375

摘要： Systems and methods for tracking human hands by performing parts based template matching using images captured from multiple viewpoints are described. One embodiment of the invention includes a processor, a reference camera, an alternate view camera, and memory containing: a hand tracking application; and a plurality of edge feature templates that are rotated and scaled versions of a finger template that includes an edge features template. In addition, the hand tracking application configures the processor to: detect at least one candidate finger in a reference frame, where each candidate finger is a grouping of pixels identified by searching the reference frame for a grouping of pixels that have image gradient orientations that match one of the plurality of edge feature templates; and verify the correct detection of a candidate finger in the reference frame by locating a grouping of pixels in an alternate view frame that correspond to the candidate finger.

摘要翻译： 描述通过使用从多个视点捕获的图像执行基于零件的模板匹配来跟踪人手的系统和方法。 本发明的一个实施例包括处理器，参考相机，备用视图相机和存储器，其包含：手跟踪应用; 以及包括边缘特征模板的手指模板的旋转和缩放版本的多个边缘特征模板。 另外，手跟踪应用将处理器配置为：在参考帧中检测至少一个候选手指，其中每个候选手指是通过搜索参考帧而识别的像素分组，所述像素分组具有匹配的图像梯度取向 多个边缘特征模板之一; 并且通过将对应于候选手指的替代视图帧中的像素分组进行定位来验证参考帧中候选手指的正确检测。

公开(公告)号：US08615108B1

公开(公告)日：2013-12-24

申请号：US13948117

申请日：2013-07-22

申请人： Imimtek, Inc.

发明人： Michele Stoppa ,  Britta Hummel ,  Carlo Dal Mutto ,  Giuliano Pasqualotto

IPC分类号： G06K9/00 ,  G06K9/48 ,  G06K9/62

CPC分类号： G06K9/00355 ,  G06K9/4671

摘要： Systems and methods for initializing motion tracking of human hands within bounded regions are disclosed. One embodiment includes: a processor; reference and alternate view cameras; and memory containing a plurality of templates that are rotated and scaled versions of a base template. In addition, a hand tracking application configures the processor to: obtain reference and alternate view frames of video data; generate a depth map; identify at least one bounded region within the reference frame of video data containing pixels having distances from the reference camera that are within a specific range of distances; determine whether any of the pixels within the at least one bounded region are part of a human hand; track the motion of the part of the human hand in a sequence of frames of video data obtained from the reference camera; and confirm that the tracked motion corresponds to a predetermined initialization gesture.

摘要翻译： 公开了用于在有界区域内初始化人手的运动跟踪的系统和方法。 一个实施例包括：处理器; 参考和备用摄像机; 以及包含基本模板的旋转和缩放版本的多个模板的存储器。 此外，手跟踪应用将处理器配置为：获得视频数据的参考视频帧和备用视图帧; 生成深度图; 识别视频数据的参考帧内的至少一个有界区域，该视频数据中包含距参考摄像机距离在特定范围内的像素的像素; 确定所述至少一个有界区域内的任何像素是否是人的手的一部分; 以从参考相机获得的视频数据的帧序列跟踪人类手部的运动; 并确认跟踪运动对应于预定的初始化手势。

公开(公告)号：US20120270653A1

公开(公告)日：2012-10-25

申请号：US13506474

申请日：2012-04-20

申请人： Nazim Kareemi ,  Abbas Rafii

发明人： Nazim Kareemi ,  Abbas Rafii

IPC分类号： A63F13/00

CPC分类号： A63F13/573 ,  A63F13/10 ,  A63F13/211 ,  A63F13/213 ,  A63F13/42 ,  A63F13/52 ,  A63F13/56 ,  A63F2300/105 ,  A63F2300/1075 ,  A63F2300/1093 ,  A63F2300/204 ,  A63F2300/6045 ,  A63F2300/646

摘要： Natural three-dimensional (xw,yw,zw,tw) gesture player interaction with a two-dimensional game application rendered on a two or three dimensional display includes mapping acquired (xw,yw,zw,tw) gesture data to virtual game-world (xv,yv,zv,tv) coordinates or vice versa, and scaling if needed. The game application is caused to render at least one image on the display responsive to the mapped and scaled (xw,yw,zw) data, where the display and game interaction is rendered from the player's perception viewpoint. The (xw,yw,zw) data preferably is acquired using spaced-apart two-dimensional cameras coupled to software to reduce the acquired images to a relatively small number of landmark points, from which player gestures may be recognized. The invention may be implemented in a handheld device such as a smart phone or tablet, which device may include a gyroscope and/or accelerometer.

摘要翻译： 与二维或三维显示器上呈现的二维游戏应用程序的自然三维（xw，yw，zw，tw）手势玩家交互包括将获取的（xw，yw，zw，tw）手势数据映射到虚拟游戏世界 （xv，yv，zv，tv）坐标，反之亦然，如果需要缩放。 导致游戏应用程序响应于映射和缩放（xw，yw，zw）数据而在显示器上呈现至少一个图像，其中从玩家的感知观点呈现显示和游戏交互。 （xw，yw，zw）数据优选地是使用耦合到软件的间隔开的二维相机来获取的，以将所获取的图像减少到相对较少数量的地标点，玩家手势可从该地图点识别。 本发明可以在诸如智能电话或平板电脑的手持设备中实现，该设备可以包括陀螺仪和/或加速度计。