公开(公告)号：US20230153522A1

公开(公告)日：2023-05-18

申请号：US17455533

申请日：2021-11-18

Applicant: ADOBE INC.

Inventor： Jaemin Cho ,  Seunghyun Yoon ,  Ajinkya Gorakhnath Kale ,  Trung Huu Bui ,  Franck Dernoncourt

IPC: G06F40/253 ,  G06K9/62 ,  G06F16/583

CPC classification number: G06F40/253 ,  G06K9/6256 ,  G06K9/6262 ,  G06F16/583

Abstract: Systems and methods for image captioning are described. One or more aspects of the systems and methods include generating a training caption for a training image using an image captioning network; encoding the training caption using a multi-modal encoder to obtain an encoded training caption; encoding the training image using the multi-modal encoder to obtain an encoded training image; computing a reward function based on the encoded training caption and the encoded training image; and updating parameters of the image captioning network based on the reward function.

公开(公告)号：US12210825B2

公开(公告)日：2025-01-28

申请号：US17455533

申请日：2021-11-18

Applicant: ADOBE INC.

Inventor： Jaemin Cho ,  Seunghyun Yoon ,  Ajinkya Gorakhnath Kale ,  Trung Huu Bui ,  Franck Dernoncourt

IPC: G06F40/253 ,  G06F16/583 ,  G06F18/21 ,  G06F18/214 ,  G06K9/62

Abstract: Systems and methods for image captioning are described. One or more aspects of the systems and methods include generating a training caption for a training image using an image captioning network; encoding the training caption using a multi-modal encoder to obtain an encoded training caption; encoding the training image using the multi-modal encoder to obtain an encoded training image; computing a reward function based on the encoded training caption and the encoded training image; and updating parameters of the image captioning network based on the reward function.