-
公开(公告)号:US20240331322A1
公开(公告)日:2024-10-03
申请号:US18190673
申请日:2023-03-27
申请人: Adobe Inc.
发明人: Cameron Smith
IPC分类号: G06T19/20 , G06V10/774 , G06V10/94 , G06V20/40 , G06V40/16
CPC分类号: G06T19/20 , G06V10/774 , G06V10/945 , G06V20/40 , G06V40/174 , G06T2200/24 , G06T2219/2021
摘要: The present disclosure relates to systems, methods, and non-transitory computer-readable media that modify digital images via scene-based editing using image understanding facilitated by artificial intelligence. For example, in one or more embodiments the disclosed systems utilize generative machine learning models to create modified digital images portraying human subjects. In particular, the disclosed systems generate modified digital images by performing infill modifications to complete a digital image or human inpainting for portions of a digital image that portrays a human. Moreover, in some embodiments, the disclosed systems perform reposing of subjects portrayed within a digital image to generate modified digital images. In addition, the disclosed systems in some embodiments perform facial expression transfer and facial expression animations to generate modified digital images or animations.
-
公开(公告)号:US12086901B2
公开(公告)日:2024-09-10
申请号:US17656907
申请日:2022-03-29
申请人: Adobe Inc.
CPC分类号: G06T11/00 , G06N3/08 , G06T7/194 , G06T7/70 , G06T2207/20021 , G06T2207/20081 , G06T2207/20084
摘要: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating painted digital images utilizing an intelligent painting process that includes progressive layering, sequential brushstroke guidance, and/or brushstroke regularization. For example, the disclosed systems utilize an image painting model to perform progressive layering to generate and apply digital brushstrokes in a progressive fashion for different layers associated with a background canvas and foreground objects. In addition, the disclosed systems utilize sequential brushstroke guidance to generate painted foreground objects by sequentially shifting through attention windows for regions of interest in a target digital image. Furthermore, the disclosed systems utilize brushstroke regularization to generate and apply an efficient brushstroke sequence to generate a painted digital image.
-
公开(公告)号:US20220122305A1
公开(公告)日:2022-04-21
申请号:US17384273
申请日:2021-07-23
申请人: Adobe Inc.
发明人: Cameron Smith , Ratheesh Kalarot , Wei-An Lin , Richard Zhang , Niloy Mitra , Elya Shechtman , Shabnam Ghadar , Zhixin Shu , Yannick Hold-Geoffrey , Nathan Carr , Jingwan Lu , Oliver Wang , Jun-Yan Zhu
摘要: An improved system architecture uses a pipeline including an encoder and a Generative Adversarial Network (GAN) including a generator neural network to generate edited images with improved speed, realism, and identity preservation. The encoder produces an initial latent space representation of an input image by encoding the input image. The generator neural network generates an initial output image by processing the initial latent space representation of the input image. The system generates an optimized latent space representation of the input image using a loss minimization technique that minimizes a loss between the input image and the initial output image. The loss is based on target perceptual features extracted from the input image and initial perceptual features extracted from the initial output image. The system outputs the optimized latent space representation of the input image for downstream use.
-
公开(公告)号:US20220122222A1
公开(公告)日:2022-04-21
申请号:US17384283
申请日:2021-07-23
申请人: Adobe Inc.
发明人: Cameron Smith , Ratheesh Kalarot , Wei-An Lin , Richard Zhang , Niloy Mitra , Elya Shechtman , Shabnam Ghadar , Zhixin Shu , Yannick Hold-Geoffrey , Nathan Carr , Jingwan Lu , Oliver Wang , Jun-Yan Zhu
摘要: An improved system architecture uses a Generative Adversarial Network (GAN) including a specialized generator neural network to generate multiple resolution output images. The system produces a latent space representation of an input image. The system generates a first output image at a first resolution by providing the latent space representation of the input image as input to a generator neural network comprising an input layer, an output layer, and a plurality of intermediate layers and taking the first output image from an intermediate layer, of the plurality of intermediate layers of the generator neural network. The system generates a second output image at a second resolution different from the first resolution by providing the latent space representation of the input image as input to the generator neural network and taking the second output image from the output layer of the generator neural network.
-
公开(公告)号:US20220121932A1
公开(公告)日:2022-04-21
申请号:US17384378
申请日:2021-07-23
申请人: Adobe Inc.
发明人: Ratheesh Kalarot , Wei-An Lin , Cameron Smith , Zhixin Shu , Baldo Faieta , Shabnam Ghadar , Jingwan Lu , Aliakbar Darabi , Jun-Yan Zhu , Niloy Mitra , Richard Zhang , Elya Shechtman
摘要: Systems and methods train an encoder neural network for fast and accurate projection into the latent space of a Generative Adversarial Network (GAN). The encoder is trained by providing an input training image to the encoder and producing, by the encoder, a latent space representation of the input training image. The latent space representation is provided as input to the GAN to generate a generated training image. A latent code is sampled from a latent space associated with the GAN and the sampled latent code is provided as input to the GAN. The GAN generates a synthetic training image based on the sampled latent code. The sampled latent code is provided as input to the encoder to produce a synthetic training code. The encoder is updated by minimizing a loss between the generated training image and the input training image, and the synthetic training code and the sampled latent code.
-
6.
公开(公告)号:US11157773B2
公开(公告)日:2021-10-26
申请号:US16802243
申请日:2020-02-26
申请人: ADOBE INC.
摘要: Images can be edited to include features similar to a different target image. An unconditional generative adversarial network (GAN) is employed to edit features of an initial image based on a constraint determined from a target image. The constraint used by the GAN is determined from keypoints or segmentation masks of the target image, and edits are made to features of the initial image based on keypoints or segmentation masks of the initial image corresponding to those of the constraint from the target image. The GAN modifies the initial image based on a loss function having a variable for the constraint. The result of this optimization process is a modified initial image having features similar to the target image subject to the constraint determined from the identified keypoints or segmentation masks.
-
公开(公告)号:US11880766B2
公开(公告)日:2024-01-23
申请号:US17384357
申请日:2021-07-23
申请人: Adobe Inc.
发明人: Cameron Smith , Ratheesh Kalarot , Wei-An Lin , Richard Zhang , Niloy Mitra , Elya Shechtman , Shabnam Ghadar , Zhixin Shu , Yannick Hold-Geoffrey , Nathan Carr , Jingwan Lu , Oliver Wang , Jun-Yan Zhu
IPC分类号: G06N3/08 , G06F3/04845 , G06F3/04847 , G06T11/60 , G06T3/40 , G06N20/20 , G06T5/00 , G06T5/20 , G06T3/00 , G06T11/00 , G06F18/40 , G06F18/211 , G06F18/214 , G06F18/21 , G06N3/045
CPC分类号: G06N3/08 , G06F3/04845 , G06F3/04847 , G06F18/211 , G06F18/214 , G06F18/2163 , G06F18/40 , G06N3/045 , G06N20/20 , G06T3/0006 , G06T3/0093 , G06T3/40 , G06T3/4038 , G06T3/4046 , G06T5/005 , G06T5/20 , G06T11/001 , G06T11/60 , G06T2207/10024 , G06T2207/20081 , G06T2207/20084 , G06T2207/20221 , G06T2210/22
摘要: An improved system architecture uses a pipeline including a Generative Adversarial Network (GAN) including a generator neural network and a discriminator neural network to generate an image. An input image in a first domain and information about a target domain are obtained. The domains correspond to image styles. An initial latent space representation of the input image is produced by encoding the input image. An initial output image is generated by processing the initial latent space representation with the generator neural network. Using the discriminator neural network, a score is computed indicating whether the initial output image is in the target domain. A loss is computed based on the computed score. The loss is minimized to compute an updated latent space representation. The updated latent space representation is processed with the generator neural network to generate an output image in the target domain.
-
8.
公开(公告)号:US20230316475A1
公开(公告)日:2023-10-05
申请号:US17709221
申请日:2022-03-30
申请人: Adobe Inc.
发明人: Cameron Smith , Wei-An Lin , Timothy M. Converse , Shabnam Ghadar , Ratheesh Kalarot , John Nack , Jingwan Lu , Hui Qu , Elya Shechtman , Baldo Faieta
CPC分类号: G06T5/50 , G06N3/0454 , G06T2207/20221 , G06T2207/20084 , G06T2207/20081
摘要: An item recommendation system receives a set of recommendable items and a request to select, from the set of recommendable items, a contrast group. The item recommendation system selects a contrast group from the set of recommendable items by applying a image modification model to the set of recommendable items. The image modification model includes an item selection model configured to determine an unbiased conversion rate for each item of the set of recommendable items and select a recommended item from the set of recommendable items having a greatest unbiased conversion rate. The image modification model includes a contrast group selection model configured to select, for the recommended item, a contrast group comprising the recommended item and one or more contrast items. The item recommendation system transmits the contrast group responsive to the request.
-
公开(公告)号:US11727614B2
公开(公告)日:2023-08-15
申请号:US17182492
申请日:2021-02-23
申请人: Adobe Inc.
发明人: Akhilesh Kumar , Baldo Faieta , Piotr Walczyszyn , Ratheesh Kalarot , Archie Bagnall , Shabnam Ghadar , Wei-An Lin , Cameron Smith , Christian Cantrell , Patrick Hebron , Wilson Chan , Jingwan Lu , Holger Winnemoeller , Sven Olsen
CPC分类号: G06T11/60 , G06N3/04 , G06T11/203
摘要: The present disclosure describes systems, methods, and non-transitory computer readable media for detecting user interactions to edit a digital image from a client device and modify the digital image for the client device by using a web-based intermediary that modifies a latent vector of the digital image and an image modification neural network to generate a modified digital image from the modified latent vector. In response to user interaction to modify a digital image, for instance, the disclosed systems modify a latent vector extracted from the digital image to reflect the requested modification. The disclosed systems further use a latent vector stream renderer (as an intermediary device) to generate an image delta that indicates a difference between the digital image and the modified digital image. The disclosed systems then provide the image delta as part of a digital stream to a client device to quickly render the modified digital image.
-
公开(公告)号:US20220122232A1
公开(公告)日:2022-04-21
申请号:US17468476
申请日:2021-09-07
申请人: Adobe Inc.
发明人: Wei-An Lin , Baldo Faieta , Cameron Smith , Elya Shechtman , Jingwan Lu , Jun-Yan Zhu , Niloy Mitra , Ratheesh Kalarot , Richard Zhang , Shabnam Ghadar , Zhixin Shu
摘要: Systems and methods generate a filtering function for editing an image with reduced attribute correlation. An image editing system groups training data into bins according to a distribution of a target attribute. For each bin, the system samples a subset of the training data based on a pre-determined target distribution of a set of additional attributes in the training data. The system identifies a direction in the sampled training data corresponding to the distribution of the target attribute to generate a filtering vector for modifying the target attribute in an input image, obtains a latent space representation of an input image, applies the filtering vector to the latent space representation of the input image to generate a filtered latent space representation of the input image, and provides the filtered latent space representation as input to a neural network to generate an output image with a modification to the target attribute.
-
-
-
-
-
-
-
-
-