公开(公告)号：US20240362272A1

公开(公告)日：2024-10-31

申请号：US18647338

申请日：2024-04-26

申请人： Twelve Labs, Inc.

发明人： Seung Joon Lee ,  Raehyuk Jung ,  Seongyun Lee ,  Minjoon Seo ,  Jaehyuk Yi

IPC分类号： G06F16/735 ,  G06F40/40 ,  G06V20/40

CPC分类号： G06F16/735 ,  G06F40/40 ,  G06V20/46

摘要： A video analysis system receives one or more queries from users of client devices. The video analysis system trains a machine-learned video encoder and/or a decoder coupled to receive video data and a prompt including a user query and generate an output for responding to the user query. A set of video embeddings are generated by extracting frame data, audio data, or text data from the video content, and applying a machine-learned video encoder to the frame data, the audio data, or the text data to generate the set of video embeddings. The video analysis system also generates a set of prompt embeddings representing at least a portion of the query in a latent space. The video analysis system applies at least a component of a machine-learned decoder to the input tensor to generate an output including a set of output embeddings.

公开(公告)号：US20230281247A1

公开(公告)日：2023-09-07

申请号：US17883493

申请日：2022-08-08

申请人： Twelve Labs, Inc.

发明人： Seung Joon LEE ,  Sung Jun KIM ,  Raehyuk JUNG ,  Haram JO

IPC分类号： G06F16/783 ,  G06V20/40 ,  G06V10/82

CPC分类号： G06F16/7837 ,  G06V20/46 ,  G06V20/49 ,  G06V10/82

摘要： In order to implement the foregoing object, an exemplary embodiment of the present disclosure discloses a video retrieval method performed by a computing device. The video retrieval method may include: identifying, by a machine learning enabled key frame detecting module having one or more encoders, key frame information for one or more video data based on one or more encoded vectors respectively generated from one or more unit video data contained in the one or more video data; segmenting the one or more video data into one or more target retrieval video data based on the identified key frame information; and generating, by a machine learning enabled retrieval vector generating module having one or more encoders, one or more retrieval video vectors respectively representing the one or more target retrieval video data.

公开(公告)号：US12045281B2

公开(公告)日：2024-07-23

申请号：US17985115

申请日：2022-11-10

申请人： Twelve Labs, Inc.

发明人： Seung Joon Lee ,  Haram Jo ,  Eunkyung Yoon ,  Soyoung Lee ,  Jae Sung Lee

IPC分类号： G06F16/70 ,  G06F16/738 ,  G06F16/783

CPC分类号： G06F16/738 ,  G06F16/784 ,  G06F16/7844

摘要： Provided is a method of providing a user interface for video retrieval performed by a computing device, the method including: receiving a search query, providing a search result interface displaying at least one search video content and search section searched by the search query, wherein the search video content and the search section are related with at least one of a plurality of video semantic search attributes corresponding to the search query, and providing a timeline view interface for the search video content when it is determined to display the search result interface according to a timeline view mode, wherein the timeline view interface has a visual characteristic indicating the degree of relevance to at least one of the plurality of video semantic search attributes corresponding to the search query.

公开(公告)号：US20240338409A1

公开(公告)日：2024-10-10

申请号：US18741545

申请日：2024-06-12

申请人： Twelve Labs, Inc.

发明人： Seung Joon Lee ,  Haram Jo ,  Eunkyung Yoon ,  Soyoung Lee ,  Jae Sung Lee

IPC分类号： G06F16/738 ,  G06F16/783

CPC分类号： G06F16/738 ,  G06F16/784 ,  G06F16/7844

摘要： Provided is a method of providing a user interface for video retrieval performed by a computing device, the method including: receiving a search query, providing a search result interface displaying at least one search video content and search section searched by the search query, wherein the search video content and the search section are related with at least one of a plurality of video semantic search attributes corresponding to the search query, and providing a timeline view interface for the search video content when it is determined to display the search result interface according to a timeline view mode, wherein the timeline view interface has a visual characteristic indicating the degree of relevance to at least one of the plurality of video semantic search attributes corresponding to the search query.

公开(公告)号：US12079273B2

公开(公告)日：2024-09-03

申请号：US17883489

申请日：2022-08-08

申请人： Twelve Labs, Inc.

发明人： Seung Joon Lee ,  Sung Jun Kim ,  Raehyuk Jung ,  Haram Jo

IPC分类号： G06F16/783 ,  G06F16/732 ,  G06V10/82 ,  G06V20/40

CPC分类号： G06F16/7837 ,  G06F16/732 ,  G06V10/82 ,  G06V20/46 ,  G06V20/49

摘要： In order to implement the foregoing object, an exemplary embodiment of the present disclosure discloses a video retrieval method performed by a computing device. The video retrieval method may include: receiving retrieval query data for one or more video data; generating a retrieval query vector based on the received retrieval query data; comparing the retrieval query vector with a plurality of video retrieval vectors each representing target retrieval video data segmented based on key frame information; and selecting one or more target retrieval video data by the result of the comparison between the retrieval query vector and the plurality of video retrieval vectors.

公开(公告)号：US20230306056A1

公开(公告)日：2023-09-28

申请号：US17883494

申请日：2022-08-08

申请人： Twelve Labs, Inc.

发明人： Seung Joon LEE ,  Sung Jun KIM ,  Raehyuk JUNG ,  Haram JO

IPC分类号： G06F16/783 ,  G06V20/40 ,  G06V10/82

CPC分类号： G06F16/7837 ,  G06V10/82 ,  G06V20/46 ,  G06V20/49

摘要： In order to implement the foregoing object, an exemplary embodiment of the present disclosure discloses a video retrieval method performed by a computing device. The video retrieval method may include: segmenting one or more video data into two or more unit video data; encoding, by one or more encoders comprised in a machine learning enabled key frame detection module, two or more unit video data; generating one or more key frame detection vectors for each of the two or more unit video data based on the result of the encoding; and generating feature vector of one or more retrieval video data based on a combination of one or more vectors among the key frame detection vectors.

公开(公告)号：US20240070197A1

公开(公告)日：2024-02-29

申请号：US17985115

申请日：2022-11-10

申请人： Twelve Labs, Inc.

发明人： Seung Joon Lee ,  Haram Jo ,  Eunkyung Yoon ,  Soyoung Lee ,  Jae Sung Lee

IPC分类号： G06F16/738 ,  G06F16/735 ,  G06F16/783

CPC分类号： G06F16/738 ,  G06F16/735 ,  G06F16/784 ,  G06F16/7844 ,  G06F16/785

摘要： Provided is a method of providing a user interface for video retrieval performed by a computing device, the method including: receiving a search query, providing a search result interface displaying at least one search video content and search section searched by the search query, wherein the search video content and the search section are related with at least one of a plurality of video semantic search attributes corresponding to the search query, and providing a timeline view interface for the search video content when it is determined to display the search result interface according to a timeline view mode, wherein the timeline view interface has a visual characteristic indicating the degree of relevance to at least one of the plurality of video semantic search attributes corresponding to the search query.

公开(公告)号：US20230281246A1

公开(公告)日：2023-09-07

申请号：US17883483

申请日：2022-08-08

申请人： Twelve Labs, Inc.

发明人： Seung Joon LEE ,  Sung Jun KIM ,  Raehyuk JUNG ,  Haram JO

IPC分类号： G06F16/783 ,  G06V20/40 ,  G06V10/82

CPC分类号： G06F16/7837 ,  G06V20/46 ,  G06V10/82

摘要： In order to implement the foregoing object, an exemplary embodiment of the present disclosure discloses a video retrieval method performed by a computing device. The video retrieval method may include: generating one or more sub video data based on one or more video data; identifying, by a machine learning enabled key frame detecting module having one or more encoders, key frame information based on one or more encoded vectors generated from the one or more sub video data; and segmenting the one or more video data into one or more retrieval video data based on the identified key frame information.

公开(公告)号：US20230281245A1

公开(公告)日：2023-09-07

申请号：US17883489

申请日：2022-08-08

申请人： Twelve Labs, Inc.

发明人： Seung Joon LEE ,  Sung Jun KIM ,  Raehyuk JUNG ,  Haram JO

IPC分类号： G06F16/732

CPC分类号： G06F16/732

摘要： In order to implement the foregoing object, an exemplary embodiment of the present disclosure discloses a video retrieval method performed by a computing device. The video retrieval method may include: receiving retrieval query data for one or more video data; generating a retrieval query vector based on the received retrieval query data; comparing the retrieval query vector with a plurality of video retrieval vectors each representing target retrieval video data segmented based on key frame information; and selecting one or more target retrieval video data by the result of the comparison between the retrieval query vector and the plurality of video retrieval vectors.