公开(公告)号：US20240386048A1

公开(公告)日：2024-11-21

申请号：US18319202

申请日：2023-05-17

Applicant: Adobe Inc.

Inventor： Bryan RUSSELL ,  Justin SALAMON ,  Daniel McKEE ,  Josef SIVIC

IPC: G06F16/438 ,  G06F16/432

Abstract: Embodiments are disclosed for an audio recommendation system trained to recommend music audio sequences for pairing with query video sequences using neural networks. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input including a query video sequence and natural language text. The disclosed systems and methods further comprise generating a fused visual-text embedding based on a visual embedding and a text embedding corresponding to the input. The disclosed systems and methods further comprise comparing audio embeddings for music audio sequences of a music audio sequences database with the fused visual-text embedding. The disclosed systems and methods further comprise determining a music audio sequence from the music audio sequences database as the recommended music audio sequence for pairing with the query video sequence based on a similarity metric calculated between an audio embedding for the music audio sequence and the fused visual-text embedding.