公开(公告)号：US20230386168A1

公开(公告)日：2023-11-30

申请号：US18192393

申请日：2023-03-29

Applicant: Beijing Baidu Netcom Science Technology Co., Ltd.

Inventor： Yipeng SUN ,  Mengjun CHENG ,  Longchao WANG ,  Xiongwei ZHU ,  Kun YAO ,  Junyu HAN ,  Jingtuo LIU ,  Errui DING ,  Jingdong WANG ,  Haifeng Wang

IPC: G06V10/42 ,  G06F16/583 ,  H04N19/176

CPC classification number: G06V10/42 ,  G06F16/5846 ,  H04N19/176

Abstract: A pre-training method for a Vision and Scene Text Aggregation model includes: acquiring a sample image-text pair; extracting a sample scene text from a sample image; inputting a sample text into a text encoding network to obtain a sample text feature; inputting the sample image and an initial sample aggregation feature into a visual encoding subnetwork and inputting the initial sample aggregation feature and the sample scene text into a scene encoding subnetwork to obtain a global image feature of the sample image and a learned sample aggregation feature; and pre-training the Vision and Scene Text Aggregation model according to the sample text feature, the global image feature of the sample image, and the learned sample aggregation feature.