公开(公告)号：US20230386208A1

公开(公告)日：2023-11-30

申请号：US17804656

申请日：2022-05-31

Applicant: ADOBE INC.

Inventor： Hailin Jin ,  Jielin Qiu ,  Zhaowen Wang ,  Trung Huu Bui ,  Franck Dernoncourt

IPC: G06V20/40 ,  G06F16/683 ,  G06V10/774 ,  G06F16/34

CPC classification number: G06V20/47 ,  G06V20/49 ,  G06F16/685 ,  G06V10/774 ,  G06F16/345

Abstract: Systems and methods for video segmentation and summarization are described. Embodiments of the present disclosure receive a video and a transcript of the video; generate visual features representing frames of the video using an image encoder; generate language features representing the transcript using a text encoder, wherein the image encoder and the text encoder are trained based on a correlation between training visual features and training language features; and segment the video into a plurality of video segments based on the visual features and the language features.