公开(公告)号：US20240395061A1

公开(公告)日：2024-11-28

申请号：US18671708

申请日：2024-05-22

Applicant: Lemon Inc. ,  Beijing Zitiao Network Technology Co., Ltd. ,  Institute of Automation Chinese Academy of Sciences

Inventor： Xiaojie JIN ,  Xingjian HE ,  Sihan CHEN ,  Fan MA ,  Zhicheng HUANG ,  Jing LIU ,  Jiashi FENG

IPC: G06V20/70 ,  G06V10/774 ,  G06V10/80 ,  G06V20/40

Abstract: The present disclosure provides a video processing method, apparatus, device, storage medium, and program product. The method includes: acquiring video data; obtaining, based on the video data, a temporal image feature with temporal information; determining, based on the temporal image feature, a target text feature in a set of text features that matches the temporal image feature; and obtaining, based on the target text feature, target text data corresponding to the video data.

公开(公告)号：US20240233350A1

公开(公告)日：2024-07-11

申请号：US18408967

申请日：2024-01-10

Applicant: Lemon Inc. ,  Beijing Zitiao Network Technology Co., Ltd.

Inventor： Xiaojie JIN ,  Fan MA ,  Jiashi FENG ,  Heng WANG ,  Jingjia HUANG

IPC: G06V10/80 ,  G06F40/284 ,  G06V10/774 ,  G06V20/40

CPC classification number: G06V10/806 ,  G06F40/284 ,  G06V10/774 ,  G06V20/46

Abstract: The embodiments of the disclosure provides a processing method, apparatus, electronic device and non-transitory computer-readable storage medium for multimodal data, wherein the method includes: obtaining data to be processed of an original modality; determining result data of a target modality corresponding to the data to be processed by processing the data to be processed with a target processing model; wherein the target processing model comprises a multimodal submodel, and the pre-training task of the multimodal submodel includes a task of locating local data that matches second modal data from first modal data; wherein when the first modal data belongs to the original modality, the second modal data belongs to the target modality; when the first modal data belongs to the target modality, the second modal data belongs to the original modality.