公开(公告)号：US20230230198A1

公开(公告)日：2023-07-20

申请号：US17576091

申请日：2022-01-14

Applicant: Adobe Inc.

Inventor： Ruiyi Zhang ,  Yufan Zhou ,  Christopher Tensmeyer ,  Jiuxiang Gu ,  Tong Yu ,  Tong Sun

IPC: G06T3/00 ,  G06T11/00 ,  G10L15/22 ,  G10L15/26 ,  G06N3/04

CPC classification number: G06T3/0056 ,  G06T11/00 ,  G10L15/22 ,  G10L15/26 ,  G06N3/04 ,  G10L2015/223

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that implement a neural network framework for interactive multi-round image generation from natural language inputs. Specifically, the disclosed systems provide an intelligent framework (i.e., a text-based interactive image generation model) that facilitates a multi-round image generation and editing workflow that comports with arbitrary input text and synchronous interaction. In particular embodiments, the disclosed systems utilize natural language feedback for conditioning a generative neural network that performs text-to-image generation and text-guided image modification. For example, the disclosed systems utilize a trained model to inject textual features from natural language feedback into a unified joint embedding space for generating text-informed style vectors. In turn, the disclosed systems can generate an image with semantically meaningful features that map to the natural language feedback. Moreover, the disclosed systems can persist these semantically meaningful features throughout a refinement process and across generated images.

公开(公告)号：US20240386621A1

公开(公告)日：2024-11-21

申请号：US18318921

申请日：2023-05-17

Applicant: Adobe Inc.

Inventor： Ruiyi Zhang ,  Yufan Zhou ,  Tong Yu ,  Tong Sun ,  Rajiv Jain ,  Jiuxiang Gu ,  Christopher Alan Tensmeyer

IPC: G06T11/00 ,  G06F40/40 ,  G06V10/74 ,  G06V10/774 ,  G06V10/82

Abstract: Techniques and systems for training and/or implementing a text-to-image generation model are provided. A pre-trained multimodal model is leveraged for avoiding slower and more labor-intensive methodologies for training a text-to-image generation model. Accordingly, images without associated text (i.e., bare images) are provided to the pre-trained multimodal model so that it can produce generated text-image pairs. The generated text-image pairs are provided to the text-to-image generation model for training and/or implementing the text-to-image generation model.

公开(公告)号：US12148119B2

公开(公告)日：2024-11-19

申请号：US17576091

申请日：2022-01-14

Applicant: Adobe Inc.

Inventor： Ruiyi Zhang ,  Yufan Zhou ,  Christopher Tensmeyer ,  Jiuxiang Gu ,  Tong Yu ,  Tong Sun

IPC: G06T5/00 ,  G06N3/04 ,  G06T3/10 ,  G06T5/60 ,  G06T11/00 ,  G06T11/80 ,  G10L15/22 ,  G10L15/26

Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that implement a neural network framework for interactive multi-round image generation from natural language inputs. Specifically, the disclosed systems provide an intelligent framework (i.e., a text-based interactive image generation model) that facilitates a multi-round image generation and editing workflow that comports with arbitrary input text and synchronous interaction. In particular embodiments, the disclosed systems utilize natural language feedback for conditioning a generative neural network that performs text-to-image generation and text-guided image modification. For example, the disclosed systems utilize a trained model to inject textual features from natural language feedback into a unified joint embedding space for generating text-informed style vectors. In turn, the disclosed systems can generate an image with semantically meaningful features that map to the natural language feedback. Moreover, the disclosed systems can persist these semantically meaningful features throughout a refinement process and across generated images.