公开(公告)号：US20220156503A1

公开(公告)日：2022-05-19

申请号：US16953049

申请日：2020-11-19

Applicant: Adobe Inc.

Inventor： Viswanathan Swaminathan ,  Stefano Petrangeli ,  Hongxiang Gu

IPC: G06K9/00 ,  G11B27/06 ,  G11B27/031 ,  G06K9/62 ,  G06N3/08

Abstract: A video summarization system generates a concatenated feature set by combining a feature set of a candidate video shot and a summarization feature set. Based on the concatenated feature set, the video summarization system calculates multiple action options of a reward function included in a trained reinforcement learning module. The video summarization system determines a reward outcome included in the multiple action options. The video summarization system modifies the summarization feature set to include the feature set of the candidate video shot by applying a particular modification indicated by the reward outcome. The video summarization system identifies video frames associated with the modified summarization feature set, and generates a summary video based on the identified video frames.

公开(公告)号：US11314970B1

公开(公告)日：2022-04-26

申请号：US16953049

申请日：2020-11-19

Applicant: Adobe Inc.

Inventor： Viswanathan Swaminathan ,  Stefano Petrangeli ,  Hongxiang Gu

IPC: G06K9/00 ,  G06N3/08 ,  G11B27/031 ,  G06K9/62 ,  G11B27/06

Abstract: A video summarization system generates a concatenated feature set by combining a feature set of a candidate video shot and a summarization feature set. Based on the concatenated feature set, the video summarization system calculates multiple action options of a reward function included in a trained reinforcement learning module. The video summarization system determines a reward outcome included in the multiple action options. The video summarization system modifies the summarization feature set to include the feature set of the candidate video shot by applying a particular modification indicated by the reward outcome. The video summarization system identifies video frames associated with the modified summarization feature set, and generates a summary video based on the identified video frames.

公开(公告)号：US10650245B2

公开(公告)日：2020-05-12

申请号：US16004170

申请日：2018-06-08

Applicant: Adobe Inc.

Inventor： Viswanathan Swaminathan ,  Hongxiang Gu

IPC: G06K9/00 ,  G11B27/031 ,  H04N21/8549 ,  G06N3/00 ,  H04N21/854 ,  H04N21/234 ,  G06K9/46 ,  G06K9/62

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating digital video summaries based on analyzing a digital video utilizing a relevancy neural network, an aesthetic neural network, and/or a generative neural network. For example, the disclosed systems can utilize an aesthetics neural network to determine aesthetics scores for frames of a digital video and a relevancy neural network to generate importance scores for frames of the digital video. Utilizing the aesthetic scores and relevancy scores, the disclosed systems can select a subset of frames and apply a generative reconstructor neural network to create a digital video reconstruction. By comparing the digital video reconstruction and the original digital video, the disclosed systems can accurately identify representative frames and flexibly generate a variety of different digital video summaries.