公开(公告)号：US20120207402A1

公开(公告)日：2012-08-16

申请号：US13455560

申请日：2012-04-25

申请人： Mihailo Stojancic ,  Prashant Ramanathan ,  Peter Wendt ,  Jose Pio Pereira

发明人： Mihailo Stojancic ,  Prashant Ramanathan ,  Peter Wendt ,  Jose Pio Pereira

IPC分类号： G06K9/40

CPC分类号： G06K9/00503 ,  G06F17/30799 ,  G06F17/30805 ,  G06F17/30814 ,  G06K9/0053 ,  G06K9/00744 ,  G06K9/4676 ,  H04N5/14

摘要： Video sequence processing is described with various filtering rules applied to extract dominant features for content based video sequence identification. Active regions are determined in video frames of a video sequence. Video frames are selected in response to temporal statistical characteristics of the determined active regions. A two pass analysis is used to detect a set of initial interest points and interest regions in the selected video frames to reduce the effective area of images that are refined by complex filters that provide accurate region characterizations resistant to image distortion for identification of the video frames in the video sequence. Extracted features and descriptors are robust with respect to image scaling, aspect ratio change, rotation, camera viewpoint change, illumination and contrast change, video compression/decompression artifacts and noise. Compact, representative signatures are generated for video sequences to provide effective query video matching and retrieval in a large video database.

摘要翻译： 使用应用于提取用于基于内容的视频序列识别的主要特征的各种滤波规则来描述视频序列处理。 在视频序列的视频帧中确定有效区域。 响应于所确定的活动区域的时间统计特性来选择视频帧。 双通道分析用于检测所选择的视频帧中的一组初始兴趣点和感兴趣区域，以减少由复杂滤波器精化的图像的有效面积，该复杂滤波器提供抵抗图像失真的精确区域表征以识别视频帧 在视频序列中。 提取的特征和描述符在图像缩放，宽高比变化，旋转，相机视点改变，照明和对比度变化，视频压缩/解压缩伪像和噪声方面是稳健的。 为视频序列生成紧凑的代表性签名，以便在大型视频数据库中提供有效的查询视频匹配和检索。

公开(公告)号：US20100303338A1

公开(公告)日：2010-12-02

申请号：US12612729

申请日：2009-11-05

申请人： Mihailo Stojancic ,  Prashant Ramanathan ,  Peter Wendt ,  Jose Pio Pereira

发明人： Mihailo Stojancic ,  Prashant Ramanathan ,  Peter Wendt ,  Jose Pio Pereira

IPC分类号： G06K9/00 ,  G06K9/46 ,  G06K9/40

CPC分类号： G06K9/00503 ,  G06F17/30799 ,  G06F17/30805 ,  G06F17/30814 ,  G06K9/0053 ,  G06K9/00744 ,  G06K9/4676 ,  H04N5/14

摘要： Video sequence processing is described with various filtering rules applied to extract dominant features for content based video sequence identification. Active regions are determined in video frames of a video sequence. Video frames are selected in response to temporal statistical characteristics of the determined active regions. A two pass analysis is used to detect a set of initial interest points and interest regions in the selected video frames to reduce the effective area of images that are refined by complex filters that provide accurate region characterizations resistant to image distortion for identification of the video frames in the video sequence. Extracted features and descriptors are robust with respect to image scaling, aspect ratio change, rotation, camera viewpoint change, illumination and contrast change, video compression/decompression artifacts and noise. Compact, representative signatures are generated for video sequences to provide effective query video matching and retrieval in a large video database.

摘要翻译： 使用应用于提取用于基于内容的视频序列识别的主要特征的各种滤波规则来描述视频序列处理。 在视频序列的视频帧中确定有效区域。 响应于所确定的活动区域的时间统计特性来选择视频帧。 双通道分析用于检测所选择的视频帧中的一组初始兴趣点和感兴趣区域，以减少由复杂滤波器精化的图像的有效面积，该复杂滤波器提供抵抗图像失真的精确区域表征以识别视频帧 在视频序列中。 提取的特征和描述符在图像缩放，宽高比变化，旋转，相机视点改变，照明和对比度变化，视频压缩/解压缩伪像和噪声方面是稳健的。 为视频序列生成紧凑的代表性签名，以便在大型视频数据库中提供有效的查询视频匹配和检索。

公开(公告)号：US09063722B2

公开(公告)日：2015-06-23

申请号：US13332482

申请日：2011-12-21

申请人： Gerald George Pechanek ,  Edwin Franklin Barry ,  Mihailo Stojancic

发明人： Gerald George Pechanek ,  Edwin Franklin Barry ,  Mihailo Stojancic

IPC分类号： G06F9/38 ,  G06F9/30 ,  G06F9/32 ,  G06F15/80

CPC分类号： G06F9/3009 ,  G06F9/32 ,  G06F9/3802 ,  G06F9/3851 ,  G06F9/3853 ,  G06F9/3885 ,  G06F9/3887 ,  G06F9/3889 ,  G06F15/8007

摘要： A control processor is used for fetching and distributing single instruction multiple data (SIMD) instructions to a plurality of processing elements (PEs). One of the SIMD instructions is a thread start (Tstart) instruction, which causes the control processor to pause its instruction fetching. A local PE instruction memory (PE Imem) is associated with each PE and contains local PE instructions for execution on the local PE. Local PE Imem fetch, decode, and execute logic are associated with each PE. Instruction path selection logic in each PE is used to select between control processor distributed instructions and local PE instructions fetched from the local PE Imem. Each PE is also initialized to receive control processor distributed instructions. In addition, local hold generation logic is associated with each PE. A PE receiving a Tstart instruction causes the instruction path selection logic to switch to fetch local PE Imem instructions.

摘要翻译： 控制处理器用于将单指令多数据（SIMD）指令提取并分发给多个处理元件（PE）。 SIMD指令之一是线程启动（Tstart）指令，它使控制处理器暂停其指令提取。 本地PE指令存储器（PE Imem）与每个PE相关联，并且包含用于在本地PE上执行的本地PE指令。 本地PE Imem提取，解码和执行逻辑与每个PE相关联。 每个PE中的指令路径选择逻辑用于在控制处理器分布式指令和从本地PE Imem获取的本地PE指令之间进行选择。 每个PE也被初始化以接收控制处理器分布式指令。 此外，本地保持生成逻辑与每个PE相关联。 接收Tstart指令的PE使指令路径选择逻辑切换到获取本地PE Imem指令。

公开(公告)号：US08959108B2

公开(公告)日：2015-02-17

申请号：US13276110

申请日：2011-10-18

申请人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

发明人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

IPC分类号： G06F17/30

CPC分类号： G06F17/30038 ,  G06F17/3002 ,  G06F17/30029 ,  G06F17/30377 ,  G06F17/3056

摘要： An efficient large scale search system for video and multi-media content using a distributed database and search, and tiered search servers is described. Selected content is stored at the distributed local database and tier1 search server(s). Content matching frequent queries, and frequent unidentified queries are cached at various levels in the search system. Content is classified using feature descriptors and geographical aspects, at feature level and in time segments. Queries not identified at clients and tier1 search server(s) are queried against tier2 or lower search server(s). Search servers use classification and geographical partitioning to reduce search cost. Methods for content tracking and local content searching are executed on clients. The client performs local search, monitoring and/or tracking of the query content with the reference content and local search with a database of reference fingerprints. This shifts the content search workload from central servers to the distributed monitoring clients.

摘要翻译： 描述了使用分布式数据库和搜索以及分层搜索服务器的视频和多媒体内容的高效大规模搜索系统。 所选内容存储在分布式本地数据库和tier1搜索服务器上。 内容匹配频繁查询和频繁的不明查询在搜索系统的各个级别进行缓存。 内容使用特征描述符和地理方面，在特征级别和时间段中进行分类。 在客户端和tier1搜索服务器上未识别的查询被查询到tier2或更低的搜索服务器。 搜索服务器使用分类和地理分区来降低搜索成本。 在客户端执行内容跟踪和本地内容搜索的方法。 客户端用参考内容和具有参考指纹数据库的本地搜索来执行查询内容的本地搜索，监视和/或跟踪。 这将内容搜索工作负载从中央服务器转移到分布式监控客户端。

公开(公告)号：US08484444B2

公开(公告)日：2013-07-09

申请号：US13037824

申请日：2011-03-01

申请人： Gerald George Pechanek ,  Mihailo Stojancic

发明人： Gerald George Pechanek ,  Mihailo Stojancic

IPC分类号： G06F9/312 ,  G06F15/80

CPC分类号： G06F15/80 ,  G06F9/30036 ,  G06F9/3824 ,  G06F9/3828 ,  G06F9/3851 ,  G06F9/3877 ,  G06F9/3885 ,  G06F9/3891

摘要： A multi-node video signal processor (VSPN) is describes that tightly couples multiple multi-cycle state machines (hardware assist units) to each processor and each memory in each node of an N node scalable array processor. VSPN memory hardware assist instructions are used to initiate multi-cycle state machine functions, to pass parameters to the multi-cycle state machines, to fetch operands from a node's memory, and to control the transfer of results from the multi-cycle state machines.

摘要翻译： 描述了一种多节点视频信号处理器（VSPN），它将多个多周期状态机（硬件辅助单元）紧密耦合到N个节点可扩展阵列处理器的每个节点的每个处理器和每个存储器。 VSPN存储器硬件辅助指令用于启动多周期状态机功能，将参数传递到多周期状态机，从节点的存储器获取操作数，并控制多循环状态机的结果传输。

公开(公告)号：US08189945B2

公开(公告)日：2012-05-29

申请号：US12612729

申请日：2009-11-05

申请人： Mihailo Stojancic ,  Prashant Ramanathan ,  Peter Wendt ,  Jose Pio Pereira

发明人： Mihailo Stojancic ,  Prashant Ramanathan ,  Peter Wendt ,  Jose Pio Pereira

IPC分类号： G06K9/00 ,  G06K9/46 ,  G06K9/40

CPC分类号： G06K9/00503 ,  G06F17/30799 ,  G06F17/30805 ,  G06F17/30814 ,  G06K9/0053 ,  G06K9/00744 ,  G06K9/4676 ,  H04N5/14

摘要： Video sequence processing is described with various filtering rules applied to extract dominant features for content based video sequence identification. Active regions are determined in video frames of a video sequence. Video frames are selected in response to temporal statistical characteristics of the determined active regions. A two pass analysis is used to detect a set of initial interest points and interest regions in the selected video frames to reduce the effective area of images that are refined by complex filters that provide accurate region characterizations resistant to image distortion for identification of the video frames in the video sequence. Extracted features and descriptors are robust with respect to image scaling, aspect ratio change, rotation, camera viewpoint change, illumination and contrast change, video compression/decompression artifacts and noise. Compact, representative signatures are generated for video sequences to provide effective query video matching and retrieval in a large video database.

摘要翻译： 使用应用于提取用于基于内容的视频序列识别的主要特征的各种滤波规则来描述视频序列处理。 在视频序列的视频帧中确定有效区域。 响应于所确定的活动区域的时间统计特性来选择视频帧。 双通道分析用于检测所选择的视频帧中的一组初始兴趣点和感兴趣区域，以减少由复杂滤波器精化的图像的有效面积，该复杂滤波器提供抵抗图像失真的精确区域表征以识别视频帧 在视频序列中。 提取的特征和描述符在图像缩放，宽高比变化，旋转，相机视点改变，照明和对比度变化，视频压缩/解压缩伪像和噪声方面是稳健的。 为视频序列生成紧凑的代表性签名，以便在大型视频数据库中提供有效的查询视频匹配和检索。

公开(公告)号：US20120095958A1

公开(公告)日：2012-04-19

申请号：US13276110

申请日：2011-10-18

申请人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

发明人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

IPC分类号： G06F17/30 ,  G06F15/16

CPC分类号： G06F17/30038 ,  G06F17/3002 ,  G06F17/30029 ,  G06F17/30377 ,  G06F17/3056

摘要： An efficient large scale search system for video and multi-media content using a distributed database and search, and tiered search servers is described. Selected content is stored at the distributed local database and tier1 search server(s). Content matching frequent queries, and frequent unidentified queries are cached at various levels in the search system. Content is classified using feature descriptors and geographical aspects, at feature level and in time segments. Queries not identified at clients and tier1 search server(s) are queried against tier2 or lower search server(s). Search servers use classification and geographical partitioning to reduce search cost. Methods for content tracking and local content searching are executed on clients. The client performs local search, monitoring and/or tracking of the query content with the reference content and local search with a database of reference fingerprints. This shifts the content search workload from central servers to the distributed monitoring clients.

摘要翻译： 描述了使用分布式数据库和搜索以及分层搜索服务器的视频和多媒体内容的高效大规模搜索系统。 所选内容存储在分布式本地数据库和tier1搜索服务器上。 内容匹配频繁查询和频繁的不明查询在搜索系统的各个级别进行缓存。 内容使用特征描述符和地理方面，在特征级别和时间段中进行分类。 在客户端和tier1搜索服务器上未识别的查询被查询到tier2或更低的搜索服务器。 搜索服务器使用分类和地理分区来降低搜索成本。 在客户端执行内容跟踪和本地内容搜索的方法。 客户端用参考内容和具有参考指纹数据库的本地搜索来执行查询内容的本地搜索，监视和/或跟踪。 这将内容搜索工作负载从中央服务器转移到分布式监控客户端。

公开(公告)号：US20150112988A1

公开(公告)日：2015-04-23

申请号：US14589366

申请日：2015-01-05

申请人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

发明人： Jose Pio Pereira ,  Shashank Merchant ,  Prashant Ramanathan ,  Sunil Suresh Kulkarni ,  Mihailo Stojancic

IPC分类号： G06F17/30

CPC分类号： G06F17/30038 ,  G06F17/3002 ,  G06F17/30029 ,  G06F17/30377 ,  G06F17/3056

摘要： An efficient large scale search system for video and multi-media content using a distributed database and search, and tiered search servers is described. Selected content is stored at the distributed local database and tier1 search server(s). Content matching frequent queries, and frequent unidentified queries are cached at various levels in the search system. Content is classified using feature descriptors and geographical aspects, at feature level and in time segments. Queries not identified at clients and tier1 search server(s) are queried against tier2 or lower search server(s). Search servers use classification and geographical partitioning to reduce search cost. Methods for content tracking and local content searching are executed on clients. The client performs local search, monitoring and/or tracking of the query content with the reference content and local search with a database of reference fingerprints. This shifts the content search workload from central servers to the distributed monitoring clients.

摘要翻译： 描述了使用分布式数据库和搜索以及分层搜索服务器的视频和多媒体内容的高效大规模搜索系统。 所选内容存储在分布式本地数据库和tier1搜索服务器上。 内容匹配频繁查询和频繁的不明查询在搜索系统的各个级别进行缓存。 内容使用特征描述符和地理方面，在特征级别和时间段中进行分类。 在客户端和tier1搜索服务器上未识别的查询被查询到tier2或更低的搜索服务器。 搜索服务器使用分类和地理分区来降低搜索成本。 在客户端执行内容跟踪和本地内容搜索的方法。 客户端用参考内容和具有参考指纹数据库的本地搜索来执行查询内容的本地搜索，监视和/或跟踪。 这将内容搜索工作负载从中央服务器转移到分布式监控客户端。

公开(公告)号：US08666152B1

公开(公告)日：2014-03-04

申请号：US12955416

申请日：2010-11-29

申请人： Prashant Ramanathan ,  Mihailo Stojancic

发明人： Prashant Ramanathan ,  Mihailo Stojancic

IPC分类号： G06K9/00

CPC分类号： G06K9/00744

摘要： Techniques are described for improving accuracy and speed of video fingerprinting and identification by using video frame color and intensity information. Multiple approaches for generating a novel signature based on video frame color information are described. One is based on spatial distribution of pixel intensities with reference to derived color groups, while another is based on thresholding a joint histogram of pixel intensity, color, and spatial coordinates. These color based signatures can be deployed either as primary video content signatures, or as hash traversal signatures used to improve speed of an initial stage of video data base search procedures. They can also be used as secondary signatures during a video signature data base search with primary video content signatures, to help disambiguate possible false positive matches, and to improve marginal matches which might not be identified otherwise.

摘要翻译： 描述了通过使用视频帧颜色和强度信息来提高视频指纹识别和识别的精度和速度的技术。 描述了基于视频帧颜色信息生成新颖签名的多种方法。 一个是基于参考衍生颜色组的像素强度的空间分布，而另一个基于阈值化像素强度，颜色和空间坐标的联合直方图。 这些基于颜色的签名可以被部署为主要视频内容签名，或者作为用于提高视频数据库搜索过程的初始阶段的速度的散列遍历签名。 在主要视频内容签名的视频签名数据库搜索期间，它们也可以用作辅助签名，以帮助消除可能的假阳性匹配，并改善可能无法识别的边缘匹配。

公开(公告)号：US08655878B1

公开(公告)日：2014-02-18

申请号：US13102479

申请日：2011-05-06

申请人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo Stojancic

发明人： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo Stojancic

IPC分类号： G06F17/30 ,  G06F7/00

CPC分类号： G06F17/30799 ,  G06F17/30023 ,  G06F17/3033 ,  G06F17/30424 ,  G06F17/30864

摘要： An architecture for a multimedia search system is described. To perform similarity matching of multimedia query frames against reference content, reference database comprising of a cluster index using cluster keys to perform similarity matching and a multimedia index to perform sequence matching is built. Methods to update and maintain the reference database that enables addition and removal of the multimedia contents, including portions of multimedia content, from the reference database in a running system are described. Hierarchical multi-level partitioning methods to organize the reference database are presented. Smart partitioning of the reference multimedia content according to the nature of the multimedia content, and according to the popularity among the social media, that supports scalable fast multimedia identification is also presented. A caching mechanism for multimedia search queries in a centralized or in a decentralized distributed system and a client based local multimedia search system enabling multimedia tracking are described.

摘要翻译： 描述了用于多媒体搜索系统的架构。 为了执行多媒体查询帧与参考内容的相似性匹配，构建了使用集群密钥进行相似性匹配的集群索引的参考数据库和执行序列匹配的多媒体索引。 描述了在运行系统中更新和维护参考数据库的方法，该参考数据库能够从参考数据库添加和删除包括多媒体内容的多媒体内容的多媒体内容。 提出了组织参考数据库的分层多级分区方法。 根据多媒体内容的性质，根据社交媒体的流行度，对参考多媒体内容进行智能划分，支持可扩展快速多媒体识别。 描述了一种用于集中式或分散式分布式系统中的多媒体搜索查询的缓存机制以及启用多媒体跟踪的基于客户端的本地多媒体搜索系统。