公开(公告)号：US11487950B2

公开(公告)日：2022-11-01

申请号：US16641256

申请日：2019-04-19

Applicant: INSTITUTE OF AUTOMATION, CHINESE ACADEMY OF SCIENCES

Inventor： Jiaming Xu ,  Yiqun Yao ,  Bo Xu

IPC: G06F40/35 ,  G06K9/62 ,  G06N3/04 ,  G06N3/08

Abstract: The method of the present disclosure includes: obtaining an image to be processed and a question text corresponding to the image; using an optimized dialogue model to encode the image into an image vector and encode the question text into a question vector; generating a state vector based on the image vector and the question vector; decoding the state vector to obtain and output an answer text. A discriminator needs to be introduced in an optimization process of the optimized dialogue model. The dialogue model and the discriminator are alternately optimized until a value of a hybrid loss function of the dialogue model and a value of a loss function of the discriminator do not decrease or fall below a preset value, thereby accomplishing the optimization process.