公开(公告)号：US11698926B2

公开(公告)日：2023-07-11

申请号：US17524862

申请日：2021-11-12

申请人： Arnab Kumar Mondal ,  Deepak Sridhar ,  Niamul Quader ,  Juwei Lu ,  Peng Dai ,  Chao Xing

发明人： Arnab Kumar Mondal ,  Deepak Sridhar ,  Niamul Quader ,  Juwei Lu ,  Peng Dai ,  Chao Xing

IPC分类号： G06F16/30 ,  G06F16/732 ,  G06N3/04 ,  G06F16/783 ,  G06V20/40

CPC分类号： G06F16/7343 ,  G06F16/783 ,  G06N3/04 ,  G06V20/40

摘要： Methods and systems are described for performing video retrieval together with video grounding. A word-based query for a video is and encoded into a query representation using a trained query encoder. One or more similar video representations are identified, from a plurality of video representations that are similar to the query representation. Each similar video representation represents a respective relevant video. A grounding is generated for each relevant video by forward propagating each respective similar video representation together with the query representation through a trained grounding module. The relevant videos or identifiers of the relevant videos are outputted together with the grounding generated for each relevant video.

公开(公告)号：US20230153352A1

公开(公告)日：2023-05-18

申请号：US17524862

申请日：2021-11-12

申请人： Arnab Kumar MONDAL ,  Deepak SRIDHAR ,  Niamul QUADER ,  Juwei LU ,  Pen DAI ,  Chao XING

发明人： Arnab Kumar MONDAL ,  Deepak SRIDHAR ,  Niamul QUADER ,  Juwei LU ,  Pen DAI ,  Chao XING

IPC分类号： G06F16/732 ,  G06F16/783 ,  G06K9/00 ,  G06N3/04

CPC分类号： G06F16/7343 ,  G06F16/783 ,  G06K9/00711 ,  G06N3/04

摘要： Methods and systems are described for performing video retrieval together with video grounding. A word-based query for a video is and encoded into a query representation using a trained query encoder. One or more similar video representations are identified, from a plurality of video representations that are similar to the query representation. Each similar video representation represents a respective relevant video. A grounding is generated for each relevant video by forward propagating each respective similar video representation together with the query representation through a trained grounding module. The relevant videos or identifiers of the relevant videos are outputted together with the grounding generated for each relevant video.

公开(公告)号：US11663268B2

公开(公告)日：2023-05-30

申请号：US17025275

申请日：2020-09-18

申请人： GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP., LTD.

发明人： Jenhao Hsiao ,  Chiuman Ho

IPC分类号： G06F16/732 ,  H04N19/109 ,  G06F16/783 ,  G06V20/40 ,  G06N3/045 ,  G06V10/764 ,  G06V10/82

CPC分类号： G06F16/7343 ,  G06F16/7837 ,  G06F16/7844 ,  G06N3/045 ,  G06V10/764 ,  G06V10/82 ,  G06V20/41 ,  G06V20/46 ,  G06V20/49 ,  H04N19/109

摘要： A method and a system for retrieving video temporal segments are provided. In the method, a video is analyzed to obtain frame feature information of the video; the frame feature information is input into an encoder to output first data relating to temporal information of the video; the first data and a retrieval description for retrieving video temporal segments of the video are input into a decoder to output second data; attention computation training is conducted according to the first data and the second data; video temporal segments of the video corresponding to the retrieval description are determined according to the attention computation training.

公开(公告)号：US20070043740A1

公开(公告)日：2007-02-22

申请号：US11505427

申请日：2006-08-17

申请人： Takeshi Saito ,  Kotaro Ise ,  Tooru Kamibayashi ,  Hideki Tsutsui ,  Satoshi Ito

发明人： Takeshi Saito ,  Kotaro Ise ,  Tooru Kamibayashi ,  Hideki Tsutsui ,  Satoshi Ito

IPC分类号： G06F17/30

CPC分类号： H04N21/632 ,  G06F16/7343 ,  G06F16/748 ,  G06F16/78 ,  G11B27/11 ,  H04H60/27 ,  H04H60/74 ,  H04H60/80 ,  H04N7/163 ,  H04N21/43615 ,  H04N21/4788 ,  H04N21/8583

摘要： An information processing terminal connectable to a WWW (World Wide Web) server via a public network includes a storage unit that stores content data including image information or sound information with identification information of the content data, an acquiring unit that acquires identification information of content data from the WWW server, a retrieving unit that retrieves content data corresponding to the identification information acquired by the acquiring unit from the storage unit, and a presenting unit that presents the content data retrieved by the retrieving unit.

摘要翻译： 经由公共网络连接到WWW（万维网）服务器的信息处理终端包括存储单元，其将包含图像信息或声音信息的内容数据与内容数据的识别信息进行存储，获取单元，获取内容数据的识别信息 来自WWW服务器的检索单元，其从存储单元检索与由获取单元获取的识别信息相对应的内容数据;以及呈现单元，其呈现由检索单元检索到的内容数据。

公开(公告)号：US20040267720A1

公开(公告)日：2004-12-30

申请号：US10609257

申请日：2003-06-27

发明人： Peiya Liu ,  Amit Chakraborty ,  Liang H. Hsu

IPC分类号： G06F007/00

CPC分类号： G06F16/40 ,  G06F16/7343

摘要： A query system for structured multimedia content retrieval comprises a query language based on logic formalism for content retrieval. The language includes query constructs and formalisms for specifying different aspects of XML documents and the constructs and formalisms are particularly adapted for spatial, temporal and visual datatypes. Certain critical specification issues in MPEG-7 XML queries are identified. An XML query language with multimedia query constructs is described which is based on a logic formalism, called path predicate calculus. In this path predicate calculus, the atomic logic formulas are element predicates rather than relation predicates in relational calculus. In this path calculus query language, queries in this calculus are equivalent to finding all proofs to existential closure of logical assertions in the form of path predicates that the tree document elements must satisfy. Spatial, temporal and visual datatypes and relationships can also be described in this formalism for content retrieval.

摘要翻译： 用于结构化多媒体内容检索的查询系统包括基于用于内容检索的逻辑形式主义的查询语言。 该语言包括用于指定XML文档的不同方面的查询结构和形式，并且构造和形式主义特别适用于空间，时间和可视数据类型。 识别MPEG-7 XML查询中的某些关键规范问题。 描述了具有多媒体查询结构的XML查询语言，其基于称为路径谓词演算的逻辑形式。 在这个路径谓词演算中，原子逻辑公式是关系演算中的元素谓词，而不是关系谓词。 在这个路径微积分查询语言中，这个演算中的查询等效于找到树状文档元素必须满足的路径谓词形式的逻辑断言的存在关闭的所有证明。 空间，时间和视觉数据类型和关系也可以用于内容检索的形式主义。

公开(公告)号：US12130891B2

公开(公告)日：2024-10-29

申请号：US17402877

申请日：2021-08-16

申请人： SAMSUNG ELECTRONICS CO., LTD.

发明人： Ning Ye ,  Zhiming Hu ,  Caleb Ryan Phillips ,  Iqbal Ismail Mohomed

IPC分类号： G06F18/22 ,  G06F16/732 ,  G06F18/214 ,  G06N20/00 ,  G06V20/40

CPC分类号： G06F18/22 ,  G06F16/7343 ,  G06F18/214 ,  G06N20/00 ,  G06V20/46 ,  G06V20/44

摘要： A method of real-time video event detection includes: obtaining, based on a natural language query, a query vector; performing multimodal feature extraction on a video stream to obtain a video vector, obtaining a similarity score by comparing the query vector to the video vector; comparing the similarity score to a predetermined threshold; and activating, based on the similarity score being above the predetermined threshold, an action trigger. The multimodal feature extraction is performed using a plurality of overlapping windows that include sequential frames of the video stream.

公开(公告)号：US10037323B2

公开(公告)日：2018-07-31

申请号：US14261523

申请日：2014-04-25

申请人： AT&T Intellectual Property I, L.P.

发明人： Arnold Chester McQuaide, Jr.

IPC分类号： G06K9/00 ,  G06F17/30 ,  H04N5/445 ,  H04N21/858 ,  H04N21/231 ,  H04N21/4722 ,  H04N21/81 ,  H04N21/472 ,  G06Q30/02 ,  G11B27/10

CPC分类号： G06F16/41 ,  G06F16/71 ,  G06F16/7343 ,  G06Q30/02 ,  G11B27/105 ,  H04N5/445 ,  H04N21/23109 ,  H04N21/47202 ,  H04N21/4722 ,  H04N21/8133 ,  H04N21/858

摘要： Methods, systems, and products index digital scenes in digital media. A uniform resource locator is assigned to each different digital scene within the digital media. The uniform resource locator uniquely identifies a resource from which each different digital scene may be retrieved. Individual scenes may thus be retrieved, thus conserving bandwidth and memory.

公开(公告)号：US20170026705A1

公开(公告)日：2017-01-26

申请号：US14808354

申请日：2015-07-24

申请人： Nuance Communications, Inc.

发明人： Peter Yeh ,  William Jarrold ,  Adwait Ratnaparkhi ,  Deepak Ramachandran ,  Peter Patel-Schneider ,  Benjamin Douglas

IPC分类号： H04N21/482 ,  G06F17/24 ,  G10L15/18 ,  H04N21/422 ,  G10L15/22 ,  H04N21/4722 ,  G06F17/30 ,  H04N21/466

CPC分类号： H04N21/4828 ,  G06F16/3329 ,  G06F16/7343 ,  G06F16/78 ,  G06F17/278 ,  G06F17/2785 ,  H04N21/4126 ,  H04N21/42203 ,  H04N21/4665 ,  H04N21/4722 ,  H04N21/4821 ,  H04N21/4826

摘要： Presenting natural-language-understanding (NLU) results can include redundancies and awkward sentence structures. In an embodiment of the present invention, a method includes, responsive to receiving a result to a NLU query, loading a matching template of a plurality of templates stored in a memory. Each template has mask fields associated with at least one property. The method compares the properties of the mask fields of each of the templates to properties of the query and properties of the result, and selects the matching template. The method further completes the matching template by inserting fields of the result into corresponding mask fields of the matching template. The method may further suppress certain mask fields of the matching template to increase brevity and improve the naturalness of the response when appropriate based on the results of the NLU query. The method further presents the completed matching template to a user via a display.

摘要翻译： 呈现自然语言理解（NLU）的结果可能包括冗余和尴尬的句子结构。 在本发明的实施例中，一种方法包括响应于将结果接收到NLU查询，加载存储在存储器中的多个模板的匹配模板。 每个模板都有与至少一个属性相关联的掩码字段。 该方法将每个模板的掩码字段的属性与查询的属性和结果的属性进行比较，并选择匹配的模板。 该方法通过将结果的字段插入匹配模板的相应掩码字段来进一步完成匹配模板。 该方法可以基于NLU查询的结果进一步抑制匹配模板的某些掩码字段以增加简洁度并且在适当时提高响应的自然度。 该方法还通过显示器向用户呈现完成的匹配模板。

公开(公告)号：US20110258295A1

公开(公告)日：2011-10-20

申请号：US13171385

申请日：2011-06-28

申请人： Takeshi SAITO ,  Kotaro ISE ,  Tooru KAMIBAYASHI ,  Hideki TSUTSUI ,  Satoshi ITO

发明人： Takeshi SAITO ,  Kotaro ISE ,  Tooru KAMIBAYASHI ,  Hideki TSUTSUI ,  Satoshi ITO

IPC分类号： G06F15/16

CPC分类号： H04N21/632 ,  G06F16/7343 ,  G06F16/748 ,  G06F16/78 ,  G11B27/11 ,  H04H60/27 ,  H04H60/74 ,  H04H60/80 ,  H04N7/163 ,  H04N21/43615 ,  H04N21/4788 ,  H04N21/8583

摘要： An information processing terminal connectable to a WWW (World Wide Web) server via a public network includes a storage unit that stores content data including image information or sound information with identification information of the content data, an acquiring unit that acquires identification information of content data from the WWW server, a retrieving unit that retrieves content data corresponding to the identification information acquired by the acquiring unit from the storage unit, and a presenting unit that presents the content data retrieved by the retrieving unit.

摘要翻译： 经由公共网络连接到WWW（万维网）服务器的信息处理终端包括存储单元，其将包含图像信息或声音信息的内容数据与内容数据的识别信息进行存储，获取单元，获取内容数据的识别信息 来自WWW服务器的检索单元，其从存储单元检索与由获取单元获取的识别信息相对应的内容数据;以及呈现单元，其呈现由检索单元检索到的内容数据。

公开(公告)号：US11995412B1

公开(公告)日：2024-05-28

申请号：US18482828

申请日：2023-10-06

申请人： Armada Systems, Inc.

发明人： Pragyana K. Mishra

IPC分类号： G06F40/40 ,  G06F16/732 ,  G06F16/735 ,  G06V10/77 ,  G06V20/40 ,  G06V20/70

CPC分类号： G06F40/40 ,  G06F16/7343 ,  G06F16/735 ,  G06V10/77 ,  G06V20/46 ,  G06V20/49 ,  G06V20/70

摘要： Disclosed are systems and methods that convert digital video data, such as two-dimensional digital video data, into a natural language text description describing the subject matter represented in the video. For example, the disclosed implementations may process video data in real-time, near real-time, or after the video data is created and generate a text-based video narrative describing the subject matter of the video. In addition, the disclosed implementations may also support a question and answer session in which a user may submit queries about the subject matter of one or more videos and the disclosed implementations will present natural language responses based on the subject matter of the video and any corresponding context.