公开(公告)号：US20250111671A1

公开(公告)日：2025-04-03

申请号：US18900457

申请日：2024-09-27

Applicant: Google LLC

Inventor： Tao Zhu ,  Jiahui Yu ,  Jingchen Feng ,  Kai Chen ,  Pooya Abolghasemi ,  Gagan Bansal ,  Jieren Xu ,  Hui Miao ,  Yaping Zhang ,  Shuchao Bi ,  Yonghui Wu ,  Claire Cui ,  Rohan Anil

IPC: G06V20/40 ,  G06F40/284 ,  G10L25/57

Abstract: Methods and systems for media item characterization based on multimodal embeddings are provided herein. A media item including a sequence of video frames is identified. A set of video embeddings representing visual features of the sequence of video frames is obtained. A set of audio embeddings representing audio features of the sequence of video frames is obtained. A set of audiovisual embeddings is generated based on the set of video embeddings and the set of audio embeddings. Each of the set of audiovisual embeddings represents a visual feature and an audio feature of a respective video frame of the sequence of video frames. One or more media characteristics associated with the media item are determined based on the set of audiovisual embeddings.