公开(公告)号：US09826216B1

公开(公告)日：2017-11-21

申请号：US15274994

申请日：2016-09-23

申请人： Aryan Hazeghi ,  Carlo Dal Mutto ,  Giulio Marin ,  Francesco Peruch ,  Michele Stoppa ,  Abbas Rafii

发明人： Aryan Hazeghi ,  Carlo Dal Mutto ,  Giulio Marin ,  Francesco Peruch ,  Michele Stoppa ,  Abbas Rafii

IPC分类号： H04N13/02 ,  H04N5/225 ,  G02B27/30 ,  G02B27/42 ,  G02B27/48 ,  G02B26/08 ,  H04N13/00

CPC分类号： H04N13/239 ,  G01B11/00 ,  G01B11/25 ,  G02B26/0808 ,  G02B27/0075 ,  G02B27/30 ,  G02B27/42 ,  G02B27/4205 ,  G02B27/425 ,  G02B27/48 ,  G03B17/54 ,  G06K9/2036 ,  G06K2209/40 ,  G06T2207/10028 ,  G06T2207/10152 ,  G06T2207/20228 ,  H04N5/2256 ,  H04N13/271 ,  H04N2013/0081

摘要： A pattern projection system includes a coherent light source, a repositionable DOE disposed to receive coherent light from said coherent light source and disposed to output at least one pattern of projectable light onto a scene to be imaged by an (x,y) two-dimensional optical acquisition system. Coherent light speckle artifacts in the projected pattern are reduced by rapidly controllably repositioning the DOE or the entire pattern projection system. Different projectable patterns are selected from a set of M patterns that are related to each other by a translation and/or rotation operation in two-dimensional cosine space. A resultant (x,y,z) depth map has improved quality and robustness due to projection of the selected patterns. Three-dimensional (x,y,z) depth data obtained from two-dimensional imaged data including despeckling is higher quality data than if projected patterns without despeckling were used.

公开(公告)号：US08854433B1

公开(公告)日：2014-10-07

申请号：US13757705

申请日：2013-02-01

申请人： Abbas Rafii

发明人： Abbas Rafii

IPC分类号： H04N7/18 ,  G06F3/01

CPC分类号： G06F3/017 ,  G06F3/011 ,  G06F3/0304 ,  G06F3/042 ,  G06F3/04815 ,  G06T15/20

摘要： An electronic device coupleable to a display screen includes a camera system that acquires optical data of a user comfortably gesturing in a user-customizable interaction zone having a z0 plane, while controlling operation of the device. Subtle gestures include hand movements commenced in a dynamically resizable and relocatable interaction zone. Preferably (x,y,z) locations in the interaction zone are mapped to two-dimensional display screen locations. Detected user hand movements can signal the device that an interaction is occurring in gesture mode. Device response includes presenting GUI on the display screen, creating user feedback including haptic feedback. User three-dimensional interaction can manipulate displayed virtual objects, including releasing such objects. User hand gesture trajectory clues enable the device to anticipate probable user intent and to appropriately update display screen renderings.

摘要翻译： 可耦合到显示屏的电子设备包括摄像机系统，其在控制设备的操作的同时，在具有z0平面的用户可自定义的交互区域中获取用户舒适的手势的光学数据。 微妙的手势包括在动态调整大小和可重定位的交互区域中开始的手动。 优选地，交互区域中的（x，y，z）位置被映射到二维显示屏幕位置。 检测到的用户手动可以向设备通知在手势模式下发生交互。 设备响应包括在显示屏幕上呈现GUI，创建包括触觉反馈的用户反馈。 用户三维交互可以操纵显示的虚拟对象，包括释放这些对象。 用户手势轨迹线索使设备能够预测可能的用户意图并适当地更新显示屏幕渲染。

公开(公告)号：US20120270653A1

公开(公告)日：2012-10-25

申请号：US13506474

申请日：2012-04-20

申请人： Nazim Kareemi ,  Abbas Rafii

发明人： Nazim Kareemi ,  Abbas Rafii

IPC分类号： A63F13/00

CPC分类号： A63F13/573 ,  A63F13/10 ,  A63F13/211 ,  A63F13/213 ,  A63F13/42 ,  A63F13/52 ,  A63F13/56 ,  A63F2300/105 ,  A63F2300/1075 ,  A63F2300/1093 ,  A63F2300/204 ,  A63F2300/6045 ,  A63F2300/646

摘要： Natural three-dimensional (xw,yw,zw,tw) gesture player interaction with a two-dimensional game application rendered on a two or three dimensional display includes mapping acquired (xw,yw,zw,tw) gesture data to virtual game-world (xv,yv,zv,tv) coordinates or vice versa, and scaling if needed. The game application is caused to render at least one image on the display responsive to the mapped and scaled (xw,yw,zw) data, where the display and game interaction is rendered from the player's perception viewpoint. The (xw,yw,zw) data preferably is acquired using spaced-apart two-dimensional cameras coupled to software to reduce the acquired images to a relatively small number of landmark points, from which player gestures may be recognized. The invention may be implemented in a handheld device such as a smart phone or tablet, which device may include a gyroscope and/or accelerometer.

摘要翻译： 与二维或三维显示器上呈现的二维游戏应用程序的自然三维（xw，yw，zw，tw）手势玩家交互包括将获取的（xw，yw，zw，tw）手势数据映射到虚拟游戏世界 （xv，yv，zv，tv）坐标，反之亦然，如果需要缩放。 导致游戏应用程序响应于映射和缩放（xw，yw，zw）数据而在显示器上呈现至少一个图像，其中从玩家的感知观点呈现显示和游戏交互。 （xw，yw，zw）数据优选地是使用耦合到软件的间隔开的二维相机来获取的，以将所获取的图像减少到相对较少数量的地标点，玩家手势可从该地图点识别。 本发明可以在诸如智能电话或平板电脑的手持设备中实现，该设备可以包括陀螺仪和/或加速度计。

公开(公告)号：US08139142B2

公开(公告)日：2012-03-20

申请号：US12004305

申请日：2007-12-20

申请人： Cyrus Bamji ,  Abbas Rafii ,  Ryan E. Crabb

发明人： Cyrus Bamji ,  Abbas Rafii ,  Ryan E. Crabb

IPC分类号： G03B13/00 ,  H04N9/07 ,  H04N5/225 ,  H04N13/02 ,  G01C3/08

CPC分类号： G01S17/89 ,  G01S17/36 ,  G01S17/936 ,  H04N5/2355 ,  H04N9/045 ,  H04N13/128 ,  H04N13/271

摘要： RGB-Z imaging systems acquire RGB data typically with a high X-Y resolution RGB pixel array, and acquire Z-depth data with an array of physically larger Z pixels having additive signal properties. In each acquired frame, RGB pixels are mapped to a corresponding Z pixel. Z image resolution is enhanced by identifying Z discontinuities and identifying corresponding RGB pixels where the Z discontinuities occur. Thus segmented data enables RGB background substitution, which preferably blends foreground pixel color and substitute background color. The segmented data also enables up-sampling in which a higher XY resolution Z image with accurate Z values is obtained. Up-sampling uses an equation set enabling assignment of accurate Z values to RGB pixels. Fixed acquisition frame rates are enabled by carefully culling bad Z data. Segmenting and up-sampling enhanced video effects and enable low cost, low Z resolution arrays to function comparably to higher quality, higher resolution Z arrays.

摘要翻译： RGB-Z成像系统通常采用高X-Y分辨率RGB像素阵列获取RGB数据，并采用具有加性信号特性的物理上较大的Z像素阵列获取Z深度数据。 在每个获取的帧中，RGB像素被映射到相应的Z像素。 通过识别Z不连续性并识别发生Z不连续性的相应RGB像素来增强Z图像分辨率。 因此，分段数据可以进行RGB背景替换，其优选地将前景像素颜色和替代背景颜色混合。 分段数据还可以进行上采样，其中获得具有精确Z值的较高XY分辨率Z图像。 上采样使用能够将精确Z值分配给RGB像素的方程组。 通过仔细剔除不良Z数据可以实现固定采集帧速率。 分段和上采样增强的视频效果，并使低成本，低Z分辨率阵列能够与更高质量，更高分辨率的Z阵列相比较。

公开(公告)号：US20110295469A1

公开(公告)日：2011-12-01

申请号：US12008430

申请日：2008-01-11

申请人： Abbas Rafii ,  Richard New ,  Sunil Acharya ,  Timothy Droz

发明人： Abbas Rafii ,  Richard New ,  Sunil Acharya ,  Timothy Droz

IPC分类号： E05F15/02 ,  G06F17/00

CPC分类号： E05F15/43 ,  E05F2015/434 ,  E05F2015/483 ,  E05Y2900/531 ,  G06T7/254 ,  G06T2207/10028 ,  G06T2207/30248

摘要： Time-of-flight (TOF) three-dimensional sensing systems are deployed on or in a motor vehicle to image contact zones associated with potential contact between an avoidable object and the vehicle or vehicle frame and/or remotely controllable motorized moving door or liftgate. An algorithm processes depth data acquired by each TOF system to determine whether an avoidable object is in the associated contact zone. If present, a control signal issues to halt or reverse the mechanism moving the door. A stored database preferably includes a depth image of the contact zone absent any object, an image of the door, and volume of the door. Database images are compared to newly acquired depth images to identify pixel sensors whose depth values are statistically unlikely to represent background or the door. Pixels within the contact zone so identified are an object, and the control signal is issued.

摘要翻译： 飞行时间（TOF）三维感测系统部署在机动车辆上或机动车辆中，以对与可避免物体与车辆或车辆框架和/或远程可控的电动移动门或升降门之间的潜在接触相关联的图像接触区进行成像。 一种算法处理由每个TOF系统获取的深度数据，以确定一个可避免的对象是否在相关联的接触区域中。 如果存在，则控制信号出现停止或反转移动门的机构。 存储的数据库优选地包括没有任何对象的接触区域的深度图像，门的图像和门的体积。 将数据库图像与新获取的深度图像进行比较，以识别其深度值在统计上不太可能代表背景或门的像素传感器。 所识别的接触区域内的像素是对象，并且发出控制信号。

公开(公告)号：US20050024324A1

公开(公告)日：2005-02-03

申请号：US10750452

申请日：2003-12-30

申请人： Carlo Tomasi ,  Abbas Rafii

发明人： Carlo Tomasi ,  Abbas Rafii

IPC分类号： G06F1/16 ,  G06F3/00 ,  G06F3/01 ,  G06F3/02 ,  G06F3/033 ,  G06F3/042 ,  G09G5/00

CPC分类号： G06F3/011 ,  G06F1/1613 ,  G06F1/1626 ,  G06F1/1632 ,  G06F1/1673 ,  G06F3/0221 ,  G06F3/0423 ,  G06F3/0426 ,  G06F3/04886 ,  G06K9/224

摘要： A system used with a virtual device inputs or transfers information to a companion device, and includes two optical systems OS1, OS2. In a structured-light embodiment, OS1 emits a fan beam plane of optical energy parallel to and above the virtual device. When a user-object penetrates the beam plane of interest, OS2 registers the event. Triangulation methods can locate the virtual contact, and transfer user-intended information to the companion system. In a non-structured active light embodiment, OS1 is preferably a digital camera whose field of view defines the plane of interest, which is illuminated by an active source of optical energy. Preferably the active source, OS1, and OS2 operate synchronously to reduce effects of ambient light. A non-structured passive light embodiment is similar except the source of optical energy is ambient light. A subtraction technique preferably enhances the signal/noise ratio. The companion device may in fact house the present invention.

摘要翻译： 与虚拟设备一起使用的系统将信息输入或传送到配套设备，并且包括两个光学系统OS1，OS2。 在结构化光实施例中，OS1发射平行于虚拟设备并且在虚拟设备之上的光能的扇形光束平面。 当用户对象穿透感兴趣的光束平面时，OS2会注册该事件。 三角测量方法可以定位虚拟联系人，并将用户意图的信息传输到配套系统。 在非结构化主动光实施例中，OS1优选地是其视场限定感兴趣平面的数字照相机，其由有效的光能源照亮。 优选地，有源源OS1和OS2同步地操作以减少环境光的影响。 非结构化无源光实施例是类似的，除了光能源是环境光。 减法技术优选地增强信噪比。 伴侣装置实际上可以容纳本发明。

公开(公告)号：US10997970B1

公开(公告)日：2021-05-04

申请号：US16947269

申请日：2020-07-27

申请人： Abbas Rafii

发明人： Abbas Rafii

IPC分类号： G10L15/06

摘要： A hearing aid system presents a hearing impaired user with customized enhanced intelligibility sound in a preferred language. The system includes a model trained with a set of source speech data representing sampling from a speech population relevant to the user. The model is also trained with a set of corresponding alternative articulation of source data, pre-defined or algorithmically constructed during an interactive session with the user. The model creates a set of selected target speech training data from the set of alternative articulation data that is preferred by the user as being satisfactorily intelligible and clear. The system includes a machine learning model, trained to shift incoming source speech data to a preferred variant of the target data that the hearing aid system presents to the user.

公开(公告)号：US09414115B1

公开(公告)日：2016-08-09

申请号：US14671282

申请日：2015-03-27

申请人： Teresa Mao ,  Abbas Rafii

发明人： Teresa Mao ,  Abbas Rafii

IPC分类号： H04N21/442 ,  H04N21/81 ,  H04N21/24 ,  H04N21/25 ,  H04N21/2668 ,  H04N21/258 ,  H04N21/466 ,  H04N21/45

CPC分类号： H04N21/44218 ,  H04N21/2407 ,  H04N21/251 ,  H04N21/25891 ,  H04N21/2668 ,  H04N21/4223 ,  H04N21/4532 ,  H04N21/4667 ,  H04N21/812

摘要： A sponsor of ads included in media content broadcast to devices by a media broadcast system for viewing by users can receive realtime feedback from users indicative of user evaluation of the presently broadcast and viewed ad. User devices anonymously acquire, process, analyze and broadcast user responses to broadcast ads viewed on the device, the responses preferably made with natural user gestures. User responses broadcast from the device are received by the media broadcast system. Ad sponsors may the customized the ad and/or future ads for the user based upon feedback and, if present, a user history profile. Broadcast ads can allow user to preselect desired ads by interacting with sponsor logos or icons presented on the device. Gesture data can be acquired, processed and broadcast to the media broadcast system for latent, incomplete, user responses, and for responses made during non-ad portions of the broadcast media.

摘要翻译： 由媒体广播系统广播到媒体广播系统的媒体内容广告的赞助者可以接收来自用户的实时反馈，指示当前广播和观看广告的用户评估。 用户设备匿名地获取，处理，分析和广播用户对在设备上观看的广播广告的响应，响应优选地以自然的用户手势进行。 从设备广播的用户响应由媒体广播系统接收。 广告赞助商可以基于反馈和（如果存在）定制用户的广告和/或将来的广告用户历史简档。 广播广告可以允许用户通过与设备上呈现的赞助商徽标或图标进行交互来预选所需的广告。 可以获取，处理手势数据并将其广播到媒体广播系统以进行潜在的，不完整的，用户响应以及在广播媒体的非广告部分期间作出的响应。

公开(公告)号：US08836768B1

公开(公告)日：2014-09-16

申请号：US13975257

申请日：2013-08-23

申请人： Abbas Rafii ,  Tony Zuccarino

发明人： Abbas Rafii ,  Tony Zuccarino

IPC分类号： H04N13/02 ,  H04N7/18 ,  G06T17/00 ,  G09G5/00 ,  G06F3/01

CPC分类号： G06F3/017 ,  G02B27/017 ,  G02B2027/0178 ,  G06F3/0304

摘要： User wearable eye glasses include a pair of two-dimensional cameras that optically acquire information for user gestures made with an unadorned user object in an interaction zone responsive to viewing displayed imagery, with which the user can interact. Glasses systems intelligently signal process and map acquired optical information to rapidly ascertain a sparse (x,y,z) set of locations adequate to identify user gestures. The displayed imagery can be created by glasses systems and presented with a virtual on-glasses display, or can be created and/or viewed off-glasses. In some embodiments the user can see local views directly, but augmented with imagery showing internet provided tags identifying and/or providing information as to viewed objects. On-glasses systems can communicate wirelessly with cloud servers and with off-glasses systems that the user can carry in a pocket or purse.

摘要翻译： 用户可佩戴的眼镜包括一对二维相机，其可以响应于观看显示的图像而光学地获取用于在交互区域中的未装饰的用户对象所进行的用户手势的信息，用户可以与之交互。 眼镜系统智能地信号处理并映射获取的光学信息，以快速确定足以识别用户手势的稀疏（x，y，z）集合。 显示的图像可以由眼镜系统创建并呈现虚拟的眼镜显示器，或者可以在眼镜之外创建和/或查看。 在一些实施例中，用户可以直接看到本地视图，但是通过显示互联网提供的标签的图像来增强识别和/或提供关于被查看对象的信息。 眼镜系统可以与云服务器和用户可以携带的口袋或钱包中的离眼镜系统进行无线通信。

公开(公告)号：US08773512B1

公开(公告)日：2014-07-08

申请号：US13507446

申请日：2012-06-29

申请人： Abbas Rafii

发明人： Abbas Rafii

IPC分类号： H04N11/02 ,  G02B5/124 ,  A63F13/06 ,  G06F3/01

CPC分类号： G06F3/017 ,  A63F13/06 ,  G02B5/124 ,  G06F3/0304 ,  G06F3/042

摘要： A portable remote control device enables user interaction with an appliance by detecting user gestures made in a hover zone, and converting the gestures to commands that are wirelessly transmitted to the appliance. The remote control device includes at least two cameras whose intersecting FOVs define a three-dimensional hover zone within which user interactions are imaged. Separately and collectively image data is analyzed to identify a relatively few user landmarks. Substantially unambiguous correspondence is established between the same landmark on each acquired image, and a three-dimensional reconstruction is made in a common coordinate system. Preferably cameras are modeled to have characteristics of pinhole cameras, enabling rectified epipolar geometric analysis to facilitate more rapid disambiguation among potential landmark points. As a result processing overhead and latency times are substantially reduced. Landmark identification and position information is convertible into commands that alter the appliance behavior as intended by the user's gesture.

摘要翻译： 便携式远程控制设备使用户能够通过检测在悬停区域中进行的用户手势，并将手势转换为无线传送到设备的命令，使用户能够与设备进行交互。 遥控装置包括至少两个相机，其相交的FOV定义用于在其中对用户交互进行成像的三维悬停区域。 分析和统一图像数据被分析以识别相对较少的用户地标。 在每个获取的图像上的相同标记之间建立基本上明确的对应关系，并且在公共坐标系中进行三维重建。 优选的是，相机被建模为具有针孔相机的特征，使矫正的对极几何分析能够有助于在潜在的地标点之间更快地消除歧义。 因此，处理开销和延迟时间大大降低。 地标识别和位置信息可转换成用户手势改变设备行为的命令。