公开(公告)号：US20220036127A1

公开(公告)日：2022-02-03

申请号：US16943511

申请日：2020-07-30

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xihui Liu ,  Quan Hung Tran ,  Jianming Zhang ,  Handong Zhao

IPC: G06K9/62 ,  G06T11/00 ,  G06K9/46

Abstract: The technology described herein is directed to a reinforcement learning based framework for training a natural media agent to learn a rendering policy without human supervision or labeled datasets. The reinforcement learning based framework feeds the natural media agent a training dataset to implicitly learn the rendering policy by exploring a canvas and minimizing a loss function. Once trained, the natural media agent can be applied to any reference image to generate a series (or sequence) of continuous-valued primitive graphic actions, e.g., sequence of painting strokes, that when rendered by a synthetic rendering environment on a canvas, reproduce an identical or transformed version of the reference image subject to limitations of an action space and the learned rendering policy.

公开(公告)号：US11809822B2

公开(公告)日：2023-11-07

申请号：US16803480

申请日：2020-02-27

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xihui Liu ,  Quan Tran ,  Jianming Zhang ,  Handong Zhao

IPC: G06F16/538 ,  G06F40/216 ,  G06F16/583 ,  G06N3/08 ,  G06F40/30 ,  G06F16/56 ,  G06F16/2457 ,  G06V30/262 ,  G06F18/22 ,  G06F18/213 ,  G06F18/214 ,  G06V30/19 ,  G06V10/75

CPC classification number: G06F40/216 ,  G06F16/24578 ,  G06F16/538 ,  G06F16/56 ,  G06F16/5854 ,  G06F18/213 ,  G06F18/214 ,  G06F18/22 ,  G06F40/30 ,  G06N3/08 ,  G06V10/75 ,  G06V30/19147 ,  G06V30/274

Abstract: Certain embodiments involve a method for generating a search result. The method includes processing devices performing operations including receiving a query having a text input by a joint embedding model trained to generate an image result. Training the joint embedding model includes accessing a set of images and textual information. Training further includes encoding the images into image feature vectors based on spatial features. Further, training includes encoding the textual information into textual feature vectors based on semantic information. Training further includes generating a set of image-text pairs based on matches between image feature vectors and textual feature vectors. Further, training includes generating a visual grounding dataset based on spatial information. Training further includes generating a set of visual-semantic joint embeddings by grounding the image-text pairs with the visual grounding dataset. Additionally, operations include generating an image result for display by the joint embedding model based on the text input.

公开(公告)号：US11574142B2

公开(公告)日：2023-02-07

申请号：US16943511

申请日：2020-07-30

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xihui Liu ,  Quan Hung Tran ,  Jianming Zhang ,  Handong Zhao

IPC: G06K9/62 ,  G06T11/00 ,  G06V10/44

Abstract: The technology described herein is directed to a reinforcement learning based framework for training a natural media agent to learn a rendering policy without human supervision or labeled datasets. The reinforcement learning based framework feeds the natural media agent a training dataset to implicitly learn the rendering policy by exploring a canvas and minimizing a loss function. Once trained, the natural media agent can be applied to any reference image to generate a series (or sequence) of continuous-valued primitive graphic actions, e.g., sequence of painting strokes, that when rendered by a synthetic rendering environment on a canvas, reproduce an identical or transformed version of the reference image subject to limitations of an action space and the learned rendering policy.

公开(公告)号：US20210271707A1

公开(公告)日：2021-09-02

申请号：US16803480

申请日：2020-02-27

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xihui Liu ,  Quan Tran ,  Jianming Zhang ,  Handong Zhao

IPC: G06F16/583 ,  G06K9/62 ,  G06K9/72 ,  G06F40/30 ,  G06F16/538 ,  G06F16/56 ,  G06F16/2457 ,  G06N3/08

Abstract: Certain embodiments involve a method for generating a search result. The method includes processing devices performing operations including receiving a query having a text input by a joint embedding model trained to generate an image result. Training the joint embedding model includes accessing a set of images and textual information. Training further includes encoding the images into image feature vectors based on spatial features. Further, training includes encoding the textual information into textual feature vectors based on semantic information. Training further includes generating a set of image-text pairs based on matches between image feature vectors and textual feature vectors. Further, training includes generating a visual grounding dataset based on spatial information. Training further includes generating a set of visual-semantic joint embeddings by grounding the image-text pairs with the visual grounding dataset. Additionally, operations include generating an image result for display by the joint embedding model based on the text input.