公开(公告)号：US20240127820A1

公开(公告)日：2024-04-18

申请号：US17967502

申请日：2022-10-17

Applicant: Adobe Inc.

Inventor： Justin Jonathan SALAMON ,  Fabian David CABA HEILBRON ,  Xue BAI ,  Aseem Omprakash AGARWALA ,  Hijung SHIN ,  Lubomira Assenova DONTCHEVA

IPC: G10L15/26 ,  G11B27/031

CPC classification number: G10L15/26 ,  G11B27/031

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for music-aware speaker diarization. In an example embodiment, one or more audio classifiers detect speech and music independently of each other, which facilitates detecting regions in an audio track that contain music but do not contain speech. These music-only regions are compared to the transcript, and any transcription and speakers that overlap in time with the music-only regions are removed from the transcript. In some embodiments, rather than having the transcript display the text from this detected music, a visual representation of the audio waveform is included in the corresponding regions of the transcript.

公开(公告)号：US20240244287A1

公开(公告)日：2024-07-18

申请号：US18154412

申请日：2023-01-13

Applicant: Adobe Inc.

Inventor： Kim Pascal PIMMEL ,  Stephen Joseph DIVERDI ,  Jiaju MA ,  Rubaiat HABIB ,  Li-Yi WEI ,  Hijung SHIN ,  Deepali ANEJA ,  John G. NELSON ,  Wilmot LI ,  Dingzeyu LI ,  Lubomira Assenova DONTCHEVA ,  Joel Richard BRANDT

IPC: H04N21/431 ,  G06F3/04812 ,  G06F3/0482 ,  H04N21/4402

CPC classification number: H04N21/4312 ,  G06F3/04812 ,  G06F3/0482 ,  H04N21/440236

Abstract: Embodiments of the present disclosure provide, a method, a system, and a computer storage media that provide mechanisms for multimedia effect addition and editing support for text-based video editing tools. The method includes generating a user interface (UI) displaying a transcript of an audio track of a video and receiving, via the UI, input identifying selection of a text segment from the transcript. The method also includes in response to receiving, via the UI, input identifying selection of a particular type of text stylization or layout for application to the text segment. The method further includes identifying a video effect corresponding to the particular type of text stylization or layout, applying the video effect to a video segment corresponding to the text segment, and applying the particular type of text stylization or layout to the text segment to visually represent the video effect in the transcript.

公开(公告)号：US20240135973A1

公开(公告)日：2024-04-25

申请号：US17967364

申请日：2022-10-17

Applicant: Adobe Inc.

Inventor： Xue BAI ,  Justin Jonathan SALAMON ,  Aseem Omprakash AGARWALA ,  Hijung SHIN ,  Haoran CAI ,  Joel Richard BRANDT ,  Lubomira Assenova DONTCHEVA ,  Cristin Ailidh Fraser

IPC: G11B27/036 ,  G06F40/166 ,  G10L15/26 ,  G10L25/57 ,  G11B27/34

CPC classification number: G11B27/036 ,  G06F40/166 ,  G10L15/26 ,  G10L25/57 ,  G11B27/34 ,  G06F3/0482

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for identifying candidate boundaries for video segments, video segment selection using those boundaries, and text-based video editing of video segments selected via transcript interactions. In an example implementation, boundaries of detected sentences and words are extracted from a transcript, the boundaries are retimed into an adjacent speech gap to a location where voice or audio activity is a minimum, and the resulting boundaries are stored as candidate boundaries for video segments. As such, a transcript interface presents the transcript, interprets input selecting transcript text as an instruction to select a video segment with corresponding boundaries selected from the candidate boundaries, and interprets commands that are traditionally thought of as text-based operations (e.g., cut, copy, paste) as an instruction to perform a corresponding video editing operation using the selected video segment.

公开(公告)号：US20240134909A1

公开(公告)日：2024-04-25

申请号：US17967703

申请日：2022-10-17

Applicant: Adobe Inc.

Inventor： Lubomira Assenova DONTCHEVA ,  Dingzeyu LI ,  Kim Pascal PIMMEL ,  Hijung SHIN ,  Hanieh DEILAMSALEHY ,  Aseem Omprakash AGARWALA ,  Joy Oakyung KIM ,  Joel Richard BRANDT ,  Cristin Ailidh Fraser

IPC: G06F16/732

CPC classification number: G06F16/732

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for a visual and text search interface used to navigate a video transcript. In an example embodiment, a freeform text query triggers a visual search for frames of a loaded video that match the freeform text query (e.g., frame embeddings that match a corresponding embedding of the freeform query), and triggers a text search for matching words from a corresponding transcript or from tags of detected features from the loaded video. Visual search results are displayed (e.g., in a row of tiles that can be scrolled to the left and right), and textual search results are displayed (e.g., in a row of tiles that can be scrolled up and down). Selecting (e.g., clicking or tapping on) a search result tile navigates a transcript interface to a corresponding portion of the transcript.

公开(公告)号：US20230043769A1

公开(公告)日：2023-02-09

申请号：US17969536

申请日：2022-10-19

Applicant: Adobe Inc.

Inventor： Seth WALKER ,  Joy O KIM ,  Aseem AGARWALA ,  Joel Richard Brandt ,  Jovan POPOVIC ,  Lubomira DONTCHEVA ,  Dingzeyu LI ,  Hijung SHIN ,  Xue Bai

IPC: G06F3/04847 ,  G06F3/0485 ,  G06F3/04845

Abstract: Embodiments are directed to techniques for interacting with a hierarchical video segmentation using a video timeline. In some embodiments, the finest level of a hierarchical segmentation identifies the smallest interaction unit of a video—semantically defined video segments of unequal duration called clip atoms, and higher levels cluster the clip atoms into coarser sets of video segments. A presented video timeline is segmented based on one of the levels, and one or more segments are selected through interactions with the video timeline. For example, a click or tap on a video segment or a drag operation dragging along the timeline snaps selection boundaries to corresponding segment boundaries defined by the level. Navigating to a different level of the hierarchy transforms the selection into coarser or finer video segments defined by the level. Any operation can be performed on selected video segments, including playing back, trimming, or editing.

公开(公告)号：US20240127858A1

公开(公告)日：2024-04-18

申请号：US17967608

申请日：2022-10-17

Applicant: Adobe Inc.

Inventor： Lubomira Assenova DONTCHEVA ,  Hijung SHIN ,  Joel Richard BRANDT ,  Joy Oakyung KIM

IPC: G11B27/031 ,  G10L15/24 ,  G10L15/26

CPC classification number: G11B27/031 ,  G10L15/24 ,  G10L15/26

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for annotating transcript text with video metadata, and including thumbnail bars in the transcript to help users select a desired portion of a video through transcript interactions. In an example embodiment, a video editing interface includes a transcript interface that presents a transcript with transcript text that is annotated to indicate corresponding portions of the video where various features were detected (e.g., annotating via text stylization of transcript text and/or labeling the transcript text with a textual representation of a corresponding detected feature class). In some embodiments, the transcript interface displays a visual representation of detected non-speech audio or pauses (e.g., a sound bar) and/or video thumbnails corresponding to each line of transcript text (e.g., a thumbnail bar). Transcript text, soundbars, and/or thumbnail bars are selectable to identify and perform video editing operations on a corresponding video segment.

公开(公告)号：US20240126994A1

公开(公告)日：2024-04-18

申请号：US17967562

申请日：2022-10-17

Applicant: Adobe Inc.

Inventor： Hanieh DEILAMSALEHY ,  Aseem Omprakash AGARWALA ,  Haoran CAI ,  Hijung SHIN ,  Joel Richard BRANDT ,  Lubomira Assenova DONTCHEVA

IPC: G06F40/30 ,  G06F40/205 ,  H04N5/93

CPC classification number: G06F40/30 ,  G06F40/205 ,  H04N5/9305

Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for segmenting a transcript into paragraphs. In an example embodiment, a transcript is segmented to start a new paragraph whenever there is a change in speaker and/or a long pause in speech. If any remaining paragraphs are longer than a designated length or duration (e.g., 50 or 100 words), each of those paragraphs is segmented using dynamic programming to minimize a cost function that penalizes candidate paragraphs based on divergence from a target paragraph length and/or that rewards candidate paragraphs that group semantically similar sentences. As such, the transcript is visualized, segmented at the identified paragraphs.

公开(公告)号：US20210142827A1

公开(公告)日：2021-05-13

申请号：US16679013

申请日：2019-11-08

Applicant: ADOBE INC.

Inventor： William Hayes ALLEN ,  Lubomira DONTCHEVA ,  Haiqing LU ,  Zachary Platt MCCULLOUGH ,  David R. STEIN ,  Christopher NUUJA ,  Benoit AMBRY ,  Joel Richard Brandt ,  Cristin Ailidh FRASER ,  Joy Oakyung KIM ,  Hijung SHIN

IPC: G11B27/034 ,  G11B27/036 ,  G06F3/0484

Abstract: Systems and methods provide for capturing and presenting content creation tools of an application used in a video. Application data from the application for the duration of the video is received. The application data includes data identifiers and time markers corresponding to user interaction with an application in a video. The application data is processed to detect tool identifiers identifying tools used in the video based on the data identifiers. For each a tool identifier, a tool label and a corresponding time in the timeline is determined. A tool record storing the tool labels and the corresponding times in association with the video is generated. When a viewer requests to watch the video, the tool record is presented to the viewer in conjunction with the video.