公开(公告)号：US08560972B2

公开(公告)日：2013-10-15

申请号：US10914922

申请日：2004-08-10

Applicant: Andrew D Wilson

Inventor： Andrew D Wilson

IPC: G06F3/033 ,  G06F3/041 ,  G06F3/00 ,  G06F3/01 ,  G09G5/00

CPC classification number: G06F3/0425 ,  G06F3/04883

Abstract: Disclosed is a unique system and method that facilitates gesture-based interaction with a user interface. The system involves an object sensing configured to include a sensing plane vertically or horizontally located between at least two imaging components on one side and a user on the other. The imaging components can acquire input images taken of a view of and through the sensing plane. The images can include objects which are on the sensing plane and/or in the background scene as well as the user as he interacts with the sensing plane. By processing the input images, one output image can be returned which shows the user objects that are in contact with the plane. Thus, objects located at a particular depth can be readily determined. Any other objects located beyond can be “removed” and not seen in the output image.

Abstract translation: 公开了一种促进与用户界面的基于手势的交互的独特的系统和方法。 该系统涉及被配置为包括垂直或水平位于一侧的至少两个成像组件与另一侧上的用户之间的感测平面的物体感测。 成像组件可以获取通过感测平面拍摄的输入图像。 图像可以包括在感测平面和/或背景场景以及用户与感测平面交互的物体。 通过处理输入图像，可以返回一个输出图像，显示与飞机接触的用户对象。 因此，可以容易地确定位于特定深度的物体。 位于超出的任何其他对象可以被“删除”，并且在输出图像中看不到。

公开(公告)号：US08427424B2

公开(公告)日：2013-04-23

申请号：US12241281

申请日：2008-09-30

Applicant: Björn U. Hartmann ,  Andrew D. Wilson ,  Hrvoje Benko ,  Meredith J. Morris

Inventor： Björn U. Hartmann ,  Andrew D. Wilson ,  Hrvoje Benko ,  Meredith J. Morris

IPC: G09G5/00 ,  G06F3/02

CPC classification number: G06F17/241 ,  G06F3/0325 ,  G06F3/03543 ,  G06F3/0416 ,  G06F3/0421 ,  G06F3/04812 ,  G06F3/0482 ,  G06F3/04845 ,  G06F3/04883 ,  G06F3/04886 ,  G06F17/242 ,  G06F17/30017 ,  G06F2203/04808

Abstract: An interaction management module (IMM) is described for allowing users to engage an interactive surface in a collaborative environment using various input devices, such as keyboard-type devices and mouse-type devices. The IMM displays digital objects on the interactive surface that are associated with the devices in various ways. The digital objects can include input display interfaces, cursors, soft-key input mechanisms, and so on. Further, the IMM provides a mechanism for establishing a frame of reference for governing the placement of each cursor on the interactive surface. Further, the IMM provides a mechanism for allowing users to make a digital copy of a physical article placed on the interactive surface. The IMM also provides a mechanism which duplicates actions taken on the digital copy with respect to the physical article, and vice versa.

Abstract translation: 描述了交互管理模块（IMM），用于允许用户使用各种输入设备（例如键盘型设备和鼠标型设备）在协作环境中参与交互式表面。 IMM以各种方式显示与设备相关联的交互式表面上的数字对象。 数字对象可以包括输入显示界面，光标，软键输入机制等。 此外，IMM提供用于建立用于管理每个光标在交互式表面上的放置的参考框架的机制。 此外，IMM提供了允许用户制作放置在交互式表面上的物理物品的数字副本的机制。 IMM还提供了一种机制，可以复制关于物理文章对数字副本采取的操作，反之亦然。

公开(公告)号：US20120212509A1

公开(公告)日：2012-08-23

申请号：US13074041

申请日：2011-03-29

Applicant: Hrvoje Benko ,  Ricardo Jorge Jota Costa ,  Andrew D. Wilson

Inventor： Hrvoje Benko ,  Ricardo Jorge Jota Costa ,  Andrew D. Wilson

IPC: G09G5/00

CPC classification number: G03B35/00 ,  G02B27/017 ,  G06F3/01 ,  G06F3/033 ,  G06T19/006 ,  H04N13/156 ,  H04N13/275

Abstract: An interaction system is described which uses a depth camera to capture a depth image of a physical object placed on, or in vicinity to, an interactive surface. The interaction system also uses a video camera to capture a video image of the physical object. The interaction system can then generate a 3D virtual object based on the depth image and video image. The interaction system then uses a 3D projector to project the 3D virtual object back onto the interactive surface, e.g., in a mirrored relationship to the physical object. A user may then capture and manipulate the 3D virtual object in any manner. Further, the user may construct a composite model based on smaller component 3D virtual objects. The interaction system uses a projective texturing technique to present a realistic-looking 3D virtual object on a surface having any geometry.

Abstract translation: 描述了一种交互系统，其使用深度相机来捕获放置在交互式表面上或附近的物理对象的深度图像。 交互系统还使用摄像机来捕获物理对象的视频图像。 然后，交互系统可以基于深度图像和视频图像生成3D虚拟对象。 交互系统然后使用3D投影仪将3D虚拟对象投影回到交互式表面上，例如与物理对象的镜像关系。 然后，用户可以以任何方式捕获和操纵3D虚拟对象。 此外，用户可以基于较小的组件3D虚拟对象来构建复合模型。 交互系统使用投影纹理技术在具有任何几何形状的表面上呈现逼真的3D虚拟对象。

公开(公告)号：US08115732B2

公开(公告)日：2012-02-14

申请号：US12428492

申请日：2009-04-23

Applicant: Andrew D. Wilson ,  Michael J. Sinclair

Inventor： Andrew D. Wilson ,  Michael J. Sinclair

IPC: G06G5/00

CPC classification number: G06F3/017 ,  G06F3/0304 ,  G06F3/04815 ,  G06F3/04845

Abstract: Virtual controllers for visual displays are described. In one implementation, a camera captures an image of hands against a background. The image is segmented into hand areas and background areas. Various hand and finger gestures isolate parts of the background into independent areas, which are then assigned control parameters for manipulating the visual display. Multiple control parameters can be associated with attributes of multiple independent areas formed by two hands, for advanced control including simultaneous functions of clicking, selecting, executing, horizontal movement, vertical movement, scrolling, dragging, rotational movement, zooming, maximizing, minimizing, executing file functions, and executing menu choices.

Abstract translation: 描述了用于视觉显示的虚拟控制器。 在一个实现中，相机在背景上拍摄手的图像。 图像被分割为手区域和背景区域。 各种手指和手指手势将背景的一部分隔离成独立的区域，然后分配控制参数以操纵视觉显示。 多个控制参数可以与由双手形成的多个独立区域的属性相关联，用于高级控制，包括点击，选择，执行，水平移动，垂直移动，滚动，拖动，旋转移动，缩放，最大化，最小化，执行的同时功能 文件功能和执行菜单选项。

公开(公告)号：US20110271216A1

公开(公告)日：2011-11-03

申请号：US12772864

申请日：2010-05-03

Applicant: Andrew D. Wilson

Inventor： Andrew D. Wilson

IPC: G06F3/048

CPC classification number: G06F3/04812 ,  G06F3/0481 ,  G06F3/04855 ,  G06F3/0488 ,  G06F2203/04808

Abstract: Different techniques of processing user interactions with a computing system are described. In one implementation, an interactive display is configured to depict a graphical user interface which includes a plurality of different types of user interface elements (e.g., button-type element, scroll bar-type element). A user may use one or more user input object (e.g., finger, hand, stylus) to simultaneously interact with the interactive display. A plurality of different user input processing methods are used to process user inputs received by the graphical user interface differently and in accordance with the types of the user interface elements which are displayed. The processing of the user inputs is implemented to determine whether the user inputs control the respective user interface elements. The processing may determine whether the user inputs activate and/or manipulate the displayed user interface elements in but one example.

Abstract translation: 描述了处理与计算系统的用户交互的不同技术。 在一个实现中，交互式显示被配置为描绘包括多个不同类型的用户界面元素（例如，按钮类型元素，滚动条型元素）的图形用户界面。 用户可以使用一个或多个用户输入对象（例如，手指，手，手写笔）来同时与交互式显示器交互。 使用多个不同的用户输入处理方法来处理由图形用户界面接收的用户输入，并根据所显示的用户界面元素的类型进行不同的处理。 执行用户输入的处理以确定用户是否输入控制相应的用户界面元素。 该处理可以在一个示例中确定用户是否输入激活和/或操纵所显示的用户界面元素。

公开(公告)号：US08022942B2

公开(公告)日：2011-09-20

申请号：US11658000

申请日：2007-01-25

Applicant: Steven N. Bathiche ,  Andrew D. Wilson

Inventor： Steven N. Bathiche ,  Andrew D. Wilson

IPC: G06F3/042

CPC classification number: G06F3/04886 ,  G06F3/0238 ,  G06F3/0421 ,  G06F2203/04109 ,  G06F2203/04809

Abstract: A dynamic projected user interface device is disclosed, that includes a projector, a projection controller, and an imaging sensor. The projection controller is configured to receive instructions from a computing device, and to provide display images via the projector onto display surfaces. The display images are indicative of a first set of input controls when the computing device is in a first operating context, and a second set of input controls when the computing device is in a second operating context. The imaging sensor is configured to optically detect physical contacts with the one or more display surfaces.

Abstract translation: 公开了一种动态投影用户界面装置，其包括投影仪，投影控制器和成像传感器。 投影控制器被配置为从计算设备接收指令，并且经由投影仪将显示图像提供到显示表面上。 当计算设备处于第一操作环境中时，显示图像指示第一组输入控件，当计算设备处于第二操作上下文中时，显示图像指示第二组输入控件。 成像传感器被配置为光学地检测与一个或多个显示表面的物理接触。

公开(公告)号：US07970870B2

公开(公告)日：2011-06-28

申请号：US11166364

申请日：2005-06-24

Applicant: Kenneth P. Hinckley ,  Andrew D. Wilson

Inventor： Kenneth P. Hinckley ,  Andrew D. Wilson

IPC: G06F15/177 ,  G06F3/00 ,  G06F3/01 ,  G06F3/041 ,  G09G5/00

CPC classification number: H04L67/08 ,  G06F3/038 ,  G06F2203/0384 ,  H04M1/7253

Abstract: A unique system and method that facilitates extending input/output capabilities for resource deficient mobile devices and interactions between multiple heterogeneous devices is provided. The system and method involve an interactive surface to which the desired mobile devices can be connected. The interactive surface can provide an enhanced display space and customization controls for mobile devices that lack adequate displays and input capabilities. In addition, the interactive surface can be employed to permit communication and interaction between multiple mobile devices that otherwise are unable to interact with each other. When connected to the interactive surface, the mobile devices can share information, view information from their respective devices, and store information to the interactive surface. Furthermore, the interactive surface can resume activity states of mobile devices that were previously communicating upon re-connection to the surface.

Abstract translation: 提供了一种独特的系统和方法，有助于扩展资源不足的移动设备的输入/输出能力以及多个异构设备之间的交互。 该系统和方法涉及可以连接期望的移动设备的交互式表面。 交互式表面可以为缺少足够的显示和输入功能的移动设备提供增强的显示空间和定制控件。 此外，可以采用交互式表面以允许否则不能彼此交互的多个移动设备之间的通信和交互。 当连接到交互式表面时，移动设备可以共享信息，从相应设备查看信息，并将信息存储到交互式表面。 此外，交互式表面可以恢复以前在重新连接到表面时通信的移动设备的活动状态。

公开(公告)号：US07907128B2

公开(公告)日：2011-03-15

申请号：US12110032

申请日：2008-04-25

Applicant: Steven Bathiche ,  David Joshua Kurlander ,  Andrew D. Wilson ,  Christina Summer Chen ,  Joel P. Dehlin

Inventor： Steven Bathiche ,  David Joshua Kurlander ,  Andrew D. Wilson ,  Christina Summer Chen ,  Joel P. Dehlin

IPC: G06F3/042 ,  G06F3/01

CPC classification number: A63F13/65 ,  A63F3/00643 ,  A63F9/183 ,  A63F13/00 ,  A63F13/213 ,  A63F2300/1075 ,  A63F2300/1087 ,  A63F2300/69 ,  G06F3/011 ,  G06F3/016 ,  G06F3/0425 ,  G06F3/04815

Abstract: An interactive table has a display surface on which a physical object is disposed. A camera within the interactive table responds to infrared (IR) light reflected from the physical object enabling a location of the physical object on the display surface to be determined, so that the physical object appear part of a virtual environment displayed thereon. The physical object can be passive or active. An active object performs an active function, e.g., it can be self-propelled to move about on the display surface, or emit light or sound, or vibrate. The active object can be controlled by a user or the processor. The interactive table can project an image through a physical object on the display surface so the image appears part of the object. A virtual entity is preferably displayed at a position (and a size) to avoid visually interference with any physical object on the display surface.

Abstract translation: 交互式表具有设置物理对象的显示面。 交互式表中的相机响应于从物理对象反射的红外（IR）光，从而能够确定显示表面上的物理对象的位置，使得物理对象显示在其上显示的虚拟环境的一部分。 物理对象可以是被动的或主动的。 活动对象执行主动功能，例如，其可以自推进在显示表面上移动，或发出光或声音或振动。 活动对象可由用户或处理器控制。 交互式表格可以通过显示表面上的物理对象投影图像，从而将图像显示在对象的一部分中。 虚拟实体优选地在位置（和尺寸）处显示，以避免与显示表面上的任何物理对象的视觉干扰。

公开(公告)号：US07755517B2

公开(公告)日：2010-07-13

申请号：US11471009

申请日：2006-06-20

Applicant: Raman K. Sarin ,  Andrew D. Wilson

Inventor： Raman K. Sarin ,  Andrew D. Wilson

IPC: G08G1/123

CPC classification number: G01C21/20 ,  G01C17/30

Abstract: A mobile device is provided that includes a position determination mechanism, and a data store of locations including positions for each location. The mobile device is configured to determine its own position and, based on the position of the mobile device, which location is preferred. Upon that determination, the mobile device is configured to orient a pointer in the direction of the preferred location such that a user can move in the direction of the pointer and ultimately arrive at the preferred location.

Abstract translation: 提供一种移动设备，其包括位置确定机构和包括每个位置的位置的位置的数据存储。 移动设备被配置为确定其自己的位置，并且基于移动设备的位置优选哪个位置。 在该确定的情况下，移动设备被配置为沿着优选位置的方向定向指针，使得用户可以在指针的方向上移动并且最终到达优选位置。

公开(公告)号：US07665041B2

公开(公告)日：2010-02-16

申请号：US10396653

申请日：2003-03-25

Applicant: Andrew D. Wilson ,  Nuria M. Oliver

Inventor： Andrew D. Wilson ,  Nuria M. Oliver

IPC: G06F3/048 ,  G06F3/033

CPC classification number: G06F3/017 ,  G06K9/00355

Abstract: Architecture for implementing a perceptual user interface. The architecture comprises alternative modalities for controlling computer application programs and manipulating on-screen objects through hand gestures or a combination of hand gestures and verbal commands. The perceptual user interface system includes a tracking component that detects object characteristics of at least one of a plurality of objects within a scene, and tracks the respective object. Detection of object characteristics is based at least in part upon image comparison of a plurality of images relative to a course mapping of the images. A seeding component iteratively seeds the tracking component with object hypotheses based upon the presence of the object characteristics and the image comparison. A filtering component selectively removes the tracked object from the object hypotheses and/or at least one object hypothesis from the set of object hypotheses based upon predetermined removal criteria.

Abstract translation: 用于实现感性用户界面的体系结构。 该架构包括用于控制计算机应用程序和通过手势操纵屏幕对象或手势和语言命令的组合的替代模式。 感知用户界面系统包括跟踪部件，其检测场景内的多个对象中的至少一个的对象特征，并跟踪相应的对象。 对象特征的检测至少部分地基于多个图像相对于图像的路线映射的图像比较。 基于对象特征和图像比较的存在，种子组件将跟踪组件迭代地种子化对象假设。 过滤组件基于预定的去除标准，从目标假设集合中选择性地从对象假设和/或至少一个对象假设中移除跟踪对象。