公开(公告)号：US20090074300A1

公开(公告)日：2009-03-19

申请号：US12210519

申请日：2008-09-15

申请人： Jonathan J. Hull ,  Berna Erol ,  Jorge Moraleda

发明人： Jonathan J. Hull ,  Berna Erol ,  Jorge Moraleda

IPC分类号： G06K9/62

CPC分类号： G06F17/30247 ,  G06F17/30876 ,  G06K9/00463 ,  G06K9/00993 ,  G06K9/6227 ,  G06K9/6256 ,  G06K9/6262

摘要： A MMR system for newspaper publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The image registration unit includes an indexing unit for generating images adapted to the environment and capabilities of the image capture device. The indexing unit also automatically adapts the configuration of the plurality of recognition units and index tables based upon image queries applied to the plurality of recognition and index tables. The plurality of recognition units and index tables are configured based on content they reference, recognition algorithm used or other factors. The present invention also includes a number of novel methods including a method for generating blurry images for device training; a method for generating classifiers; a method for selecting a pass classifier; and a method for modifying a hypothesizing and testing classifier on unrecognized images.

摘要翻译： 用于报纸出版的MMR系统包括多个移动设备，MMR网关，MMR匹配单元和MMR发布者。 MMR匹配单元从MMR网关接收图像查询，并将其发送到一个或多个识别单元，以识别包括文档，页面和页面上的位置的结果。 图像注册单元包括索引单元，用于生成适于图像采集设备的环境和能力的图像。 索引单元还基于应用于多个识别和索引表的图像查询自动地适配多个识别单元和索引表的配置。 基于它们所引用的内容，所使用的识别算法或其他因素来配置多个识别单元和索引表。 本发明还包括许多新颖的方法，包括用于产生用于装置训练的模糊图像的方法; 一种生成分类器的方法; 选择通过分类器的方法; 以及在未识别的图像上修改假设和测试分类器的方法。

公开(公告)号：US20090016615A1

公开(公告)日：2009-01-15

申请号：US12059583

申请日：2008-03-31

申请人： Jonathan J. Hull ,  Berna Erol ,  Shigeharu Uda ,  Qifa Ke

发明人： Jonathan J. Hull ,  Berna Erol ,  Shigeharu Uda ,  Qifa Ke

IPC分类号： G06K9/64

CPC分类号： G06K9/00449 ,  G06K9/00463

摘要： The present invention uses invisible junctions which are a set of local features unique to every page of the electronic document to match the captured image to a part of an electronic document. The present invention includes: an image capture device, a feature extraction and recognition system and database. When an electronic document is printed, the feature extraction and recognition system captures an image of the document page. The features in the captured image are then extracted, indexed and stored in the database. Given a query image, the features in the query image are extracted and compared against those stored in the database to identify the query image. The feature extraction and recognition system of the present invention is integrated into a multifunction peripheral. This allows the feature extraction and recognition system to be used in conjunction with other modules to provide security and annotation applications.

摘要翻译： 本发明使用作为电子文档的每一页特有的一组局部特征的不可见结，以将捕获的图像与电子文档的一部分相匹配。 本发明包括：图像捕获装置，特征提取和识别系统和数据库。 当打印电子文档时，特征提取和识别系统捕获文档页面的图像。 然后将捕获的图像中的特征提取，索引并存储在数据库中。 给定查询图像，提取查询图像中的特征并将其与存储在数据库中的特征进行比较，以识别查询图像。 本发明的特征提取和识别系统被集成到多功能外围设备中。 这允许特征提取和识别系统与其他模块结合使用以提供安全性和注释应用。

公开(公告)号：US08856108B2

公开(公告)日：2014-10-07

申请号：US12210540

申请日：2008-09-15

申请人： Berna Erol ,  Jonathan J. Hull ,  Jorge Moraleda

发明人： Berna Erol ,  Jonathan J. Hull ,  Jorge Moraleda

IPC分类号： G06F7/00 ,  G06F17/30 ,  H04N1/21 ,  H04N1/00 ,  G06K9/00 ,  G06K9/62

CPC分类号： G06F17/30247 ,  G06F17/30876 ,  G06K9/00463 ,  G06K9/6217 ,  G06K9/6227 ,  H04N1/00244 ,  H04N1/00307 ,  H04N1/00312 ,  H04N1/2179 ,  H04N1/2191 ,  H04N2201/0039

摘要： A MMR system for newspaper publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR matching unit also includes a result combiner coupled to each of the recognition units to receive recognition results. The result combiner produces a list of most likely results and associated confidence scores. This list of results is sent by the result combiner back to the MMR gateway for presentation on the mobile device. The result combiner uses the quality predictor as an input in deciding which results are best.

摘要翻译： 用于报纸出版的MMR系统包括多个移动设备，MMR网关，MMR匹配单元和MMR发布者。 MMR匹配单元从MMR网关接收图像查询，并将其发送到一个或多个识别单元，以识别包括文档，页面和页面上的位置的结果。 MMR匹配单元还包括耦合到每个识别单元以接收识别结果的结果组合器。 结果组合器产生最可能的结果和相关置信得分的列表。 该结果列表由结果组合器发送回MMR网关以在移动设备上呈现。 结果组合器使用质量预测器作为输入来决定哪些结果是最佳的。

公开(公告)号：US08676810B2

公开(公告)日：2014-03-18

申请号：US12240596

申请日：2008-09-29

申请人： Jorge Moraleda ,  Berna Erol ,  Jonathan J. Hull

发明人： Jorge Moraleda ,  Berna Erol ,  Jonathan J. Hull

IPC分类号： G06F17/30

CPC分类号： G06F17/30247 ,  G06F17/30876 ,  G06F21/6218 ,  G06F21/78 ,  G06F21/80 ,  G06F2221/2115 ,  G06K9/00463 ,  G06K9/00993 ,  G06K9/6262

摘要： An MMR system for processing image queries across index tables with unequal priority comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page, and the location on the page. The MMR matching unit includes a dispatcher, a plurality of recognition units, and index tables, as well as an image registration unit. In one embodiment, the system includes an MMR matching plug-in installed on the mobile device. The present invention also includes methods for processing image queries across index tables of unequal priority and updating a high priority index based on received or projected image queries.

摘要翻译： 用于处理具有不等优先级的索引表的图像查询的MMR系统包括多个移动设备，预处理服务器或MMR网关以及MMR匹配单元，并且可以包括MMR发布者。 MMR匹配单元从预处理服务器或MMR网关接收图像查询，并将其发送到一个或多个识别单元，以识别包括文档，页面和页面上的位置的结果。 MMR匹配单元包括调度器，多个识别单元和索引表，以及图像配准单元。 在一个实施例中，系统包括安装在移动设备上的MMR匹配插件。 本发明还包括用于处理跨越不等优先级的索引表的图像查询的方法，并且基于接收或投影的图像查询更新高优先级索引。

公开(公告)号：US20130188864A1

公开(公告)日：2013-07-25

申请号：US13729458

申请日：2012-12-28

申请人： Jorge Moraleda ,  Berna Erol ,  Jonathan J. Hull

发明人： Jorge Moraleda ,  Berna Erol ,  Jonathan J. Hull

IPC分类号： G06K9/80

CPC分类号： G06K9/80 ,  G06F17/30247 ,  G06F17/3028 ,  G06F17/30858 ,  G06F17/30876 ,  G06K9/00463 ,  G06K9/00993 ,  G06K9/6262

摘要： An MMR system for searching across multiple indexes comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR matching unit includes a segmenter for segmenting received images by content type, a distributor for distributing the images to corresponding content type index tables, and an integrator for integrating recognition results. The result is returned to the mobile device via the pre-processing server or MMR gateway. The present invention also includes a number of novel methods including a method for processing content-type specific image queries and for processing queries across multiple indexes.

公开(公告)号：US20090100050A1

公开(公告)日：2009-04-16

申请号：US12340124

申请日：2008-12-19

申请人： Berna Erol ,  Jorge Moraleda ,  Jonathan J. Hull

发明人： Berna Erol ,  Jorge Moraleda ,  Jonathan J. Hull

IPC分类号： G06F17/30 ,  G10L15/00 ,  G06K9/46

CPC分类号： G06F17/30247 ,  G06F17/30026 ,  G06F17/30038 ,  G06F17/30047 ,  G06F17/30876 ,  G06K9/00463 ,  G06K9/00979 ,  G06K9/036 ,  G06K9/6217 ,  G06K9/6227 ,  G06K9/6262 ,  G06K9/6292

摘要： The mobile device includes a client that has a number of modules, and the MMR Gateway and MMR matching unit are implemented as a server that has a number of modules. The implementation of the MMR system as a client and a server is advantageous because the modules may be distributed among the client and the server in a variety of configurations. The present invention includes a capture module, a preprocessing module, a feature extraction module, a retrieval module, a send message module, an action module, a prediction module, a feedback module, a sending module, an MMR database, a streaming module, an e-mail module, a voice recognition system and an audio database. These modules and systems are operational upon the client or the server. In one embodiment, the client includes only the capture module with the remaining modules operational on the server. In a second embodiment, the server includes the action module with the remaining modules operational on the client.

摘要翻译： 移动设备包括具有多个模块的客户端，并且MMR网关和MMR匹配单元被实现为具有多个模块的服务器。 作为客户机和服务器的MMR系统的实现是有利的，因为模块可以以各种配置分布在客户端和服务器之间。 本发明包括捕获模块，预处理模块，特征提取模块，检索模块，发送消息模块，动作模块，预测模块，反馈模块，发送模块，MMR数据库，流模块， 电子邮件模块，语音识别系统和音频数据库。 这些模块和系统可在客户端或服务器上运行。 在一个实施例中，客户端仅包括捕获模块，其余模块在服务器上可操作。 在第二实施例中，服务器包括动作模块，其余的模块在客户端上运行。

公开(公告)号：US20090067726A1

公开(公告)日：2009-03-12

申请号：US12210532

申请日：2008-09-15

申请人： Berna Erol ,  Emilio R. Antunez ,  Landry Huet ,  Jonathan J. Hull ,  Jorge Moraleda

发明人： Berna Erol ,  Emilio R. Antunez ,  Landry Huet ,  Jonathan J. Hull ,  Jorge Moraleda

IPC分类号： G06K9/48

CPC分类号： G06F17/30247 ,  G06F17/30876 ,  G06K9/00463 ,  G06K9/00993 ,  G06K9/6256 ,  G06K2009/3291 ,  H04N1/00307 ,  Y10S707/915

摘要： A MMR system for newspaper publishing comprises a plurality of mobile devices, an MMR gateway, an MMR matching unit and an MMR publisher. The MMR matching unit receives an image query from the MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR system also includes a quality predictor as a plug-in installed on the mobile device to filter images before they are included as part of a retrieval request or as part of the MMR matching unit. The quality predictor comprises an input for receiving recognition algorithm information, a vector calculator, a score generator and a scoring module. The quality predictor receives as inputs an image query, context information and device parameters, and generates an outputs a recognizability score. The present invention also includes a method for generating robustness features.

摘要翻译： 用于报纸出版的MMR系统包括多个移动设备，MMR网关，MMR匹配单元和MMR发布者。 MMR匹配单元从MMR网关接收图像查询，并将其发送到一个或多个识别单元，以识别包括文档，页面和页面上的位置的结果。 MMR系统还包括安装在移动设备上的插件的质量预测器，以在作为检索请求的一部分被包括在内的过滤图像或者作为MMR匹配单元的一部分。 质量预测器包括用于接收识别算法信息的输入，向量计算器，得分生成器和评分模块。 质量预测器作为输入接收图像查询，上下文信息和设备参数，并产生可识别得分的输出。 本发明还包括一种用于产生鲁棒性特征的方法。

公开(公告)号：US20130346431A1

公开(公告)日：2013-12-26

申请号：US13914417

申请日：2013-06-10

申请人： Berna Erol ,  Jonathan J. Hull

发明人： Berna Erol ,  Jonathan J. Hull

IPC分类号： G06F17/30

CPC分类号： G06F17/30386 ,  G06F17/30247 ,  G06F17/30876 ,  G06K9/00463 ,  G06K9/00979 ,  G06K9/036 ,  G06K9/228 ,  G06K9/6293

摘要： An MMR system includes an analytics module for monitoring and analyzing the creation and usage of visual content search. The analytics module is part of an MMR gateway and provides a mechanism for administrators to determine the usage patterns and other information about users of the MMR system. The analytics module comprises a monitoring module, a usage database, an access analysis module and a presentation module. The monitoring module collects information about the retrieval request, image queries, context information and recognition results. The usage database stores the information collected by the monitoring module.

摘要翻译： MMR系统包括用于监视和分析可视内容搜索的创建和使用的分析模块。 分析模块是MMR网关的一部分，并为管理员提供了一种机制，用于确定MMR系统用户的使用模式和其他信息。 分析模块包括监视模块，使用数据库，访问分析模块和呈现模块。 监控模块收集有关检索请求，图像查询，上下文信息和识别结果的信息。 使用数据库存储监视模块收集的信息。

公开(公告)号：US08489987B2

公开(公告)日：2013-07-16

申请号：US12265502

申请日：2008-11-05

申请人： Berna Erol ,  Jonathan J. Hull

发明人： Berna Erol ,  Jonathan J. Hull

IPC分类号： G06F17/00

CPC分类号： G06F17/30386 ,  G06F17/30247 ,  G06F17/30876 ,  G06K9/00463 ,  G06K9/00979 ,  G06K9/036 ,  G06K9/228 ,  G06K9/6293

摘要： An MMR system includes an analytics module for monitoring and analyzing the creation and usage of visual content search. The analytics module is part of a MMR gateway and provides a mechanism for administrators to determine the usage patterns and other information about users of the MMR system. This information is fed back to other components of the MMR system to improve its operation. The analytics module comprises a monitoring module, a usage database, an access analysis module and a presentation module. The monitoring module is coupled to other components of the MMR gateway and collects information about retrieval request, image queries, context information and recognition results. The usage database stores the information collected by the monitoring module. The access analysis module performs queries on the usage database and analyzes the results. The output of the access analysis module is provided to the presentation module for presentation or output to other systems and users.

摘要翻译： MMR系统包括用于监视和分析可视内容搜索的创建和使用的分析模块。 分析模块是MMR网关的一部分，并为管理员提供了一种机制，用于确定MMR系统用户的使用模式和其他信息。 该信息被反馈给MMR系统的其他组件以改善其操作。 分析模块包括监视模块，使用数据库，访问分析模块和呈现模块。 监控模块耦合到MMR网关的其他组件，并收集关于检索请求，图像查询，上下文信息和识别结果的信息。 使用数据库存储监视模块收集的信息。 访问分析模块对使用数据库执行查询并分析结果。 访问分析模块的输出被提供给呈现模块，以呈现或输出到其他系统和用户。

公开(公告)号：US08369655B2

公开(公告)日：2013-02-05

申请号：US12240590

申请日：2008-09-29

申请人： Jorge Moraleda ,  Berna Erol ,  Jonathan J. Hull

发明人： Jorge Moraleda ,  Berna Erol ,  Jonathan J. Hull

IPC分类号： G06K9/00 ,  G06K9/54 ,  G06K9/60

CPC分类号： G06F17/30247 ,  G06F17/30876 ,  G06K9/00463 ,  G06K9/00993 ,  G06K9/6262

摘要： An MMR system for searching across multiple indexes comprises a plurality of mobile devices, a pre-processing server or MMR gateway, and an MMR matching unit, and may include an MMR publisher. The MMR matching unit receives an image query from the pre-processing server or MMR gateway and sends it to one or more of the recognition units to identify a result including a document, the page and the location on the page. The MMR matching unit includes a segmenter for segmenting received images by content type, a distributor for distributing the images to corresponding content type index tables, and an integrator for integrating recognition results. The result is returned to the mobile device via the pre-processing server or MMR gateway. The present invention also includes a number of novel methods including a method for processing content-type specific image queries and for processing queries across multiple indexes.