Invention Application
- Patent Title: METHOD FOR TRAINING IMAGE-TEXT MATCHING MODEL, COMPUTING DEVICE, AND STORAGE MEDIUM
-
Application No.: US17943458Application Date: 2022-09-13
-
Publication No.: US20230005284A1Publication Date: 2023-01-05
- Inventor: Feng HE , Qi WANG , Hu YANG , Shuai CHEN , Zhifan FENG , Chunguang CHAI
- Applicant: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.
- Applicant Address: CN BEIJING
- Assignee: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.
- Current Assignee: BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD.
- Current Assignee Address: CN BEIJING
- Priority: CN202111101658.8 20210918
- Main IPC: G06V30/19
- IPC: G06V30/19 ; G06F16/583

Abstract:
A computer-implemented method is provided. The method includes: obtaining a sample text and a sample image corresponding to the sample text; labeling a true semantic tag for the sample text according to a first preset rule; obtaining a text feature representation of the sample text and a predicted semantic tag output by a text coding sub-model; obtaining an image feature representation of the sample image output by an image coding sub-model; calculating a first loss based on the true semantic tag and the predicted semantic tag; calculating a contrast loss based on the text feature representation of the sample text and the image feature representation of the sample image; adjusting parameters of the text coding sub-model based on the first loss and the contrast loss; and adjusting parameters of the image coding sub-model based on the contrast loss.
Information query