公开(公告)号：US20240303882A1

公开(公告)日：2024-09-12

申请号：US18350876

申请日：2023-07-12

Applicant: Salesforce, Inc.

Inventor： Shu Zhang ,  Xinyi Yang ,  Yihao Feng ,  Ran Xu ,  Ning Yu ,  Chia-Chih Chen

IPC: G06T11/60 ,  G06T5/00

CPC classification number: G06T11/60 ,  G06T5/70 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Embodiments described herein provide a feedback based instructional image editing framework that employs a diffusion process to follow user instruction for image editing. A diffusion model is fine-tuned using a reward model, which may be trained via human annotation. The training of the reward model may be done by having the image editing model output a number of images, which a human annotator ranks based on their alignment with the original image and a given instruction.

公开(公告)号：US20240104809A1

公开(公告)日：2024-03-28

申请号：US18161680

申请日：2023-01-30

Applicant: Salesforce, Inc.

Inventor： Ning Yu ,  Chia-Chih Chen ,  Zeyuan Chen ,  Caiming Xiong ,  Juan Carlos Niebles Duque ,  Ran Xu ,  Rui Meng

IPC: G06T11/60 ,  G06F40/106 ,  G06F40/126 ,  G06N20/00 ,  G06T9/00

CPC classification number: G06T11/60 ,  G06F40/106 ,  G06F40/126 ,  G06N20/00 ,  G06T9/00 ,  G06T2200/24 ,  G06T2210/12

Abstract: Embodiments described herein provide systems and methods for multimodal layout generations for digital publications. The system may receive as inputs, a background image, one or more foreground texts, and one or more foreground images. Feature representations of the background image may be generated. The foreground inputs may be input to a layout generator which has cross attention to the background image feature representations in order to generate a layout comprising of bounding box parameters for each input item. A composite layout may be generated based on the inputs and generated bounding boxes. The resulting composite layout may then be displayed on a user interface.