公开(公告)号：US09510044B1

公开(公告)日：2016-11-29

申请号：US13327359

申请日：2011-12-15

申请人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Oleksiy Bolgarov ,  Prashant Ramanathan ,  Shashank Merchant ,  Mihailo Stojancic

发明人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Oleksiy Bolgarov ,  Prashant Ramanathan ,  Shashank Merchant ,  Mihailo Stojancic

IPC分类号： H04H60/32 ,  H04N21/44 ,  H04N21/8352 ,  H04H60/37 ,  H04H60/56 ,  H04H60/59

CPC分类号： H04N21/44008 ,  G06F21/10 ,  G06F2221/0704 ,  G06K9/00758 ,  H04H20/106 ,  H04H60/37 ,  H04H60/56 ,  H04H60/59 ,  H04H2201/90 ,  H04N21/44204 ,  H04N21/8352 ,  H04N21/8456

摘要： Content segmentation, categorization and identification methods are described. Content tracking approaches are illustrated that are suitable for large scale deployment. Time-aligned applications such as multi-language selection, customized advertisements, second screen services and content monitoring applications can be economically deployed at large scales. A client performs fingerprinting, scene change detection, audio turn detection, and logo detection on incoming video and gathers database search results, logos and text to identify and segment video streams into content, promos, and commercials. A learning engine is configured to learn rules for optimal identification and segmentation at each client for each channel and program. Content sensed at the client site is tracked with reduced computation and applications are executed with timing precision. A user interface for time-aligned publishing of content and subsequent usage and interaction on one or more displays is also described.

摘要翻译： 描述内容分割，分类和识别方法。 说明适用于大规模部署的内容跟踪方法。 诸如多语言选择，定制广告，第二屏幕服务和内容监控应用之类的时间对应的应用可以大规模经济地部署。 客户端对进入的视频执行指纹识别，场景变化检测，音频转向检测和标识检测，并收集数据库搜索结果，徽标和文本，以将视频流识别和分割为内容，宣传和广告。 学习引擎配置为每个通道和程序的每个客户端学习最佳识别和分段的规则。 通过减少的计算跟踪在客户端站点处感测的内容，并以时序精度执行应用程序。 还描述了用于在一个或多个显示器上时间对齐地发布内容和随后的使用和交互的用户界面。

公开(公告)号：US08959108B2

公开(公告)日：2015-02-17

申请号：US13276110

申请日：2011-10-18

申请人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

发明人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

IPC分类号： G06F17/30

CPC分类号： G06F17/30038 ,  G06F17/3002 ,  G06F17/30029 ,  G06F17/30377 ,  G06F17/3056

摘要： An efficient large scale search system for video and multi-media content using a distributed database and search, and tiered search servers is described. Selected content is stored at the distributed local database and tier1 search server(s). Content matching frequent queries, and frequent unidentified queries are cached at various levels in the search system. Content is classified using feature descriptors and geographical aspects, at feature level and in time segments. Queries not identified at clients and tier1 search server(s) are queried against tier2 or lower search server(s). Search servers use classification and geographical partitioning to reduce search cost. Methods for content tracking and local content searching are executed on clients. The client performs local search, monitoring and/or tracking of the query content with the reference content and local search with a database of reference fingerprints. This shifts the content search workload from central servers to the distributed monitoring clients.

摘要翻译： 描述了使用分布式数据库和搜索以及分层搜索服务器的视频和多媒体内容的高效大规模搜索系统。 所选内容存储在分布式本地数据库和tier1搜索服务器上。 内容匹配频繁查询和频繁的不明查询在搜索系统的各个级别进行缓存。 内容使用特征描述符和地理方面，在特征级别和时间段中进行分类。 在客户端和tier1搜索服务器上未识别的查询被查询到tier2或更低的搜索服务器。 搜索服务器使用分类和地理分区来降低搜索成本。 在客户端执行内容跟踪和本地内容搜索的方法。 客户端用参考内容和具有参考指纹数据库的本地搜索来执行查询内容的本地搜索，监视和/或跟踪。 这将内容搜索工作负载从中央服务器转移到分布式监控客户端。

公开(公告)号：US20120095958A1

公开(公告)日：2012-04-19

申请号：US13276110

申请日：2011-10-18

申请人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

发明人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

IPC分类号： G06F17/30 ,  G06F15/16

CPC分类号： G06F17/30038 ,  G06F17/3002 ,  G06F17/30029 ,  G06F17/30377 ,  G06F17/3056

摘要： An efficient large scale search system for video and multi-media content using a distributed database and search, and tiered search servers is described. Selected content is stored at the distributed local database and tier1 search server(s). Content matching frequent queries, and frequent unidentified queries are cached at various levels in the search system. Content is classified using feature descriptors and geographical aspects, at feature level and in time segments. Queries not identified at clients and tier1 search server(s) are queried against tier2 or lower search server(s). Search servers use classification and geographical partitioning to reduce search cost. Methods for content tracking and local content searching are executed on clients. The client performs local search, monitoring and/or tracking of the query content with the reference content and local search with a database of reference fingerprints. This shifts the content search workload from central servers to the distributed monitoring clients.

摘要翻译： 描述了使用分布式数据库和搜索以及分层搜索服务器的视频和多媒体内容的高效大规模搜索系统。 所选内容存储在分布式本地数据库和tier1搜索服务器上。 内容匹配频繁查询和频繁的不明查询在搜索系统的各个级别进行缓存。 内容使用特征描述符和地理方面，在特征级别和时间段中进行分类。 在客户端和tier1搜索服务器上未识别的查询被查询到tier2或更低的搜索服务器。 搜索服务器使用分类和地理分区来降低搜索成本。 在客户端执行内容跟踪和本地内容搜索的方法。 客户端用参考内容和具有参考指纹数据库的本地搜索来执行查询内容的本地搜索，监视和/或跟踪。 这将内容搜索工作负载从中央服务器转移到分布式监控客户端。

公开(公告)号：US20150112988A1

公开(公告)日：2015-04-23

申请号：US14589366

申请日：2015-01-05

申请人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

发明人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

IPC分类号： G06F17/30

CPC分类号： G06F17/30038 ,  G06F17/3002 ,  G06F17/30029 ,  G06F17/30377 ,  G06F17/3056

摘要： An efficient large scale search system for video and multi-media content using a distributed database and search, and tiered search servers is described. Selected content is stored at the distributed local database and tier1 search server(s). Content matching frequent queries, and frequent unidentified queries are cached at various levels in the search system. Content is classified using feature descriptors and geographical aspects, at feature level and in time segments. Queries not identified at clients and tier1 search server(s) are queried against tier2 or lower search server(s). Search servers use classification and geographical partitioning to reduce search cost. Methods for content tracking and local content searching are executed on clients. The client performs local search, monitoring and/or tracking of the query content with the reference content and local search with a database of reference fingerprints. This shifts the content search workload from central servers to the distributed monitoring clients.

摘要翻译： 描述了使用分布式数据库和搜索以及分层搜索服务器的视频和多媒体内容的高效大规模搜索系统。 所选内容存储在分布式本地数据库和tier1搜索服务器上。 内容匹配频繁查询和频繁的不明查询在搜索系统的各个级别进行缓存。 内容使用特征描述符和地理方面，在特征级别和时间段中进行分类。 在客户端和tier1搜索服务器上未识别的查询被查询到tier2或更低的搜索服务器。 搜索服务器使用分类和地理分区来降低搜索成本。 在客户端执行内容跟踪和本地内容搜索的方法。 客户端用参考内容和具有参考指纹数据库的本地搜索来执行查询内容的本地搜索，监视和/或跟踪。 这将内容搜索工作负载从中央服务器转移到分布式监控客户端。

公开(公告)号：US08655878B1

公开(公告)日：2014-02-18

申请号：US13102479

申请日：2011-05-06

申请人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo Stojancic

发明人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo Stojancic

IPC分类号： G06F17/30 ,  G06F7/00

CPC分类号： G06F17/30799 ,  G06F17/30023 ,  G06F17/3033 ,  G06F17/30424 ,  G06F17/30864

摘要： An architecture for a multimedia search system is described. To perform similarity matching of multimedia query frames against reference content, reference database comprising of a cluster index using cluster keys to perform similarity matching and a multimedia index to perform sequence matching is built. Methods to update and maintain the reference database that enables addition and removal of the multimedia contents, including portions of multimedia content, from the reference database in a running system are described. Hierarchical multi-level partitioning methods to organize the reference database are presented. Smart partitioning of the reference multimedia content according to the nature of the multimedia content, and according to the popularity among the social media, that supports scalable fast multimedia identification is also presented. A caching mechanism for multimedia search queries in a centralized or in a decentralized distributed system and a client based local multimedia search system enabling multimedia tracking are described.

摘要翻译： 描述了用于多媒体搜索系统的架构。 为了执行多媒体查询帧与参考内容的相似性匹配，构建了使用集群密钥进行相似性匹配的集群索引的参考数据库和执行序列匹配的多媒体索引。 描述了在运行系统中更新和维护参考数据库的方法，该参考数据库能够从参考数据库添加和删除包括多媒体内容的多媒体内容的多媒体内容。 提出了组织参考数据库的分层多级分区方法。 根据多媒体内容的性质，根据社交媒体的流行度，对参考多媒体内容进行智能划分，支持可扩展快速多媒体识别。 描述了一种用于集中式或分散式分布式系统中的多媒体搜索查询的缓存机制以及启用多媒体跟踪的基于客户端的本地多媒体搜索系统。

公开(公告)号：US08965863B1

公开(公告)日：2015-02-24

申请号：US14151294

申请日：2014-01-09

申请人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo M. Stojancic

发明人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo M. Stojancic

IPC分类号： G06F17/00 ,  G06F17/30

CPC分类号： G06F17/30799 ,  G06F17/30023 ,  G06F17/3033 ,  G06F17/30424 ,  G06F17/30864

摘要： An architecture for a multimedia search system is described. To perform similarity matching of multimedia query frames against reference content, reference database comprising of a cluster index using cluster keys to perform similarity matching and a multimedia index to perform sequence matching is built. Methods to update and maintain the reference database that enables addition and removal of the multimedia contents, including portions of multimedia content, from the reference database in a running system are described. Hierarchical multi-level partitioning methods to organize the reference database are presented. Smart partitioning of the reference multimedia content according to the nature of the multimedia content, and according to the popularity among the social media, that supports scalable fast multimedia identification is also presented. A caching mechanism for multimedia search queries in a centralized or in a decentralized distributed system and a client based local multimedia search system enabling multimedia tracking are described.

公开(公告)号：US20150254344A1

公开(公告)日：2015-09-10

申请号：US14718218

申请日：2015-05-21

申请人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo M. Stojancic

发明人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo M. Stojancic

IPC分类号： G06F17/30

CPC分类号： G06F17/30799 ,  G06F17/30023 ,  G06F17/3033 ,  G06F17/30424 ,  G06F17/30864

摘要： An architecture for a multimedia search system is described. To perform similarity matching of multimedia query frames against reference content, reference database comprising of a cluster index using cluster keys to perform similarity matching and a multimedia index to perform sequence matching is built. Methods to update and maintain the reference database that enables addition and removal of the multimedia contents, including portions of multimedia content, from the reference database in a running system are described. Hierarchical multi-level partitioning methods to organize the reference database are presented. Smart partitioning of the reference multimedia content according to the nature of the multimedia content, and according to the popularity among the social media, that supports scalable fast multimedia identification is also presented. A caching mechanism for multimedia search queries in a centralized or in a decentralized distributed system and a client based local multimedia search system enabling multimedia tracking are described.

摘要翻译： 描述了用于多媒体搜索系统的架构。 为了执行多媒体查询帧与参考内容的相似性匹配，构建了使用集群密钥进行相似性匹配的集群索引的参考数据库和执行序列匹配的多媒体索引。 描述了在运行系统中更新和维护参考数据库的方法，该参考数据库能够从参考数据库添加和删除包括多媒体内容的多媒体内容的多媒体内容。 提出了组织参考数据库的分层多级分区方法。 根据多媒体内容的性质，根据社交媒体的流行度，对参考多媒体内容进行智能划分，支持可扩展快速多媒体识别。 描述了一种用于集中式或分散式分布式系统中的多媒体搜索查询的缓存机制以及启用多媒体跟踪的基于客户端的本地多媒体搜索系统。

公开(公告)号：US09058355B1

公开(公告)日：2015-06-16

申请号：US14151335

申请日：2014-01-09

申请人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo M. Stojancic

发明人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo M. Stojancic

IPC分类号： G06F17/30

CPC分类号： G06F17/30799 ,  G06F17/30023 ,  G06F17/3033 ,  G06F17/30424 ,  G06F17/30864

摘要： An architecture for a multimedia search system is described. To perform similarity matching of multimedia query frames against reference content, reference database comprising of a cluster index using cluster keys to perform similarity matching and a multimedia index to perform sequence matching is built. Methods to update and maintain the reference database that enables addition and removal of the multimedia contents, including portions of multimedia content, from the reference database in a running system are described. Hierarchical multi-level partitioning methods to organize the reference database are presented. Smart partitioning of the reference multimedia content according to the nature of the multimedia content, and according to the popularity among the social media, that supports scalable fast multimedia identification is also presented. A caching mechanism for multimedia search queries in a centralized or in a decentralized distributed system and a client based local multimedia search system enabling multimedia tracking are described.

摘要翻译： 描述了用于多媒体搜索系统的架构。 为了执行多媒体查询帧与参考内容的相似性匹配，构建了使用集群密钥进行相似性匹配的集群索引的参考数据库和执行序列匹配的多媒体索引。 描述了在运行系统中更新和维护参考数据库的方法，该参考数据库能够从参考数据库添加和删除包括多媒体内容的多媒体内容的多媒体内容。 提出了组织参考数据库的分层多级分区方法。 根据多媒体内容的性质，根据社交媒体的流行度，对参考多媒体内容进行智能划分，支持可扩展快速多媒体识别。 描述了一种用于集中式或分散式分布式系统中的多媒体搜索查询的缓存机制以及启用多媒体跟踪的基于客户端的本地多媒体搜索系统。

公开(公告)号：US08335786B2

公开(公告)日：2012-12-18

申请号：US12788796

申请日：2010-05-27

申请人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Shashank Merchant ,  Prashant Ramanathan ,  Pradipkumar Dineshbhai Gajjar

发明人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Shashank Merchant ,  Prashant Ramanathan ,  Pradipkumar Dineshbhai Gajjar

IPC分类号： G06F17/30

CPC分类号： G06F17/30784 ,  G06F17/30825 ,  G06K9/00758

摘要： A method is presented for large media data base query and media entry identification based on multi-level similarity search and reference-query entry correlation. Media content fingerprinting detects unique features and generates discriminative descriptors and signatures used to form preliminary reference data base. The preliminary reference data base is processed and a subset-set of it is selected to form a final reference data base. To identify a media query a fast similarity search is performed first on the reference database resulting in a preliminary set of likely matching videos. For each preliminary likely matching video a further multi-level correlation is performed which includes iterative refinement, sub-sequence merging, and final result classification.

摘要翻译： 提出了一种基于多级相似搜索和参考查询条目相关性的大型媒体数据库查询和媒体条目标识的方法。 媒体内容指纹识别检测独特的特征，并产生用于形成初步参考数据库的识别描述符和签名。 处理初步参考数据库并选择其初始参考数据库以形成最终参考数据库。 为了识别媒体查询，首先在参考数据库上执行快速相似性搜索，导致可能匹配的视频的初步集合。 对于每个初步可能的匹配视频，执行进一步的多级相关，其包括迭代细化，子序列合并和最终结果分类。

公开(公告)号：US20100306193A1

公开(公告)日：2010-12-02

申请号：US12788796

申请日：2010-05-27

申请人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Shashank Merchant ,  Prashant Ramanathan ,  Pradipkumar Dineshbhai Gajjar

发明人： Jose Pio Pereira ,  Sunil Suresh Kulkarni ,  Shashank Merchant ,  Prashant Ramanathan ,  Pradipkumar Dineshbhai Gajjar

IPC分类号： G06F17/30

CPC分类号： G06F17/30784 ,  G06F17/30825 ,  G06K9/00758

摘要： A method is presented for large media data base query and media entry identification based on multi-level similarity search and reference-query entry correlation. Media content fingerprinting detects unique features and generates discriminative descriptors and signatures used to form preliminary reference data base. The preliminary reference data base is processed and a subset-set of it is selected to form a final reference data base. To identify a media query a fast similarity search is performed first on the reference database resulting in a preliminary set of likely matching videos. For each preliminary likely matching video a further multi-level correlation is performed which includes iterative refinement, sub-sequence merging, and final result classification.

摘要翻译： 提出了一种基于多级相似搜索和参考查询条目相关性的大型媒体数据库查询和媒体条目标识的方法。 媒体内容指纹识别检测独特的特征，并产生用于形成初步参考数据库的识别描述符和签名。 处理初步参考数据库并选择其初始参考数据库以形成最终参考数据库。 为了识别媒体查询，首先在参考数据库上执行快速相似性搜索，导致可能匹配的视频的初步集合。 对于每个初步可能的匹配视频，执行进一步的多级相关，其包括迭代细化，子序列合并和最终结果分类。