Digital image completion using deep learning

    公开(公告)号:US11250548B2

    公开(公告)日:2022-02-15

    申请号:US16791939

    申请日:2020-02-14

    Applicant: Adobe Inc.

    Abstract: Digital image completion using deep learning is described. Initially, a digital image having at least one hole is received. This holey digital image is provided as input to an image completer formed with a framework that combines generative and discriminative neural networks based on learning architecture of the generative adversarial networks. From the holey digital image, the generative neural network generates a filled digital image having hole-filling content in place of holes. The discriminative neural networks detect whether the filled digital image and the hole-filling digital content correspond to or include computer-generated content or are photo-realistic. The generating and detecting are iteratively continued until the discriminative neural networks fail to detect computer-generated content for the filled digital image and hole-filling content or until detection surpasses a threshold difficulty. Responsive to this, the image completer outputs the filled digital image with hole-filling content in place of the holey digital image's holes.

    HIERARCHICAL SCALE MATCHING AND PATCH ESTIMATION FOR IMAGE STYLE TRANSFER WITH ARBITRARY RESOLUTION

    公开(公告)号:US20200349688A1

    公开(公告)日:2020-11-05

    申请号:US16930736

    申请日:2020-07-16

    Applicant: Adobe Inc.

    Abstract: A style of a digital image is transferred to another digital image of arbitrary resolution. A high-resolution (HR) content image is segmented into several low-resolution (LR) patches. The resolution of a style image is matched to have the same resolution as the LR content image patches. Style transfer is then performed on a patch-by-patch basis using, for example, a pair of feature transforms—whitening and coloring. The patch-by-patch style transfer process is then repeated at several increasing resolutions, or scale levels, of both the content and style images. The results of the style transfer at each scale level are incorporated into successive scale levels up to and including the original HR scale. As a result, style transfer can be performed with images having arbitrary resolutions to produce visually pleasing results with good spatial consistency.

    HIERARCHICAL SCALE MATCHING AND PATCH ESTIMATION FOR IMAGE STYLE TRANSFER WITH ARBITRARY RESOLUTION

    公开(公告)号:US20200258204A1

    公开(公告)日:2020-08-13

    申请号:US16271058

    申请日:2019-02-08

    Applicant: Adobe Inc.

    Abstract: A style of a digital image is transferred to another digital image of arbitrary resolution. A high-resolution (HR) content image is segmented into several low-resolution (LR) patches. The resolution of a style image is matched to have the same resolution as the LR content image patches. Style transfer is then performed on a patch-by-patch basis using, for example, a pair of feature transforms—whitening and coloring. The patch-by-patch style transfer process is then repeated at several increasing resolutions, or scale levels, of both the content and style images. The results of the style transfer at each scale level are incorporated into successive scale levels up to and including the original HR scale. As a result, style transfer can be performed with images having arbitrary resolutions to produce visually pleasing results with good spatial consistency.

    Sketch-based 3D fluid volume generation using a machine learning system

    公开(公告)号:US10650581B1

    公开(公告)日:2020-05-12

    申请号:US16185587

    申请日:2018-11-09

    Applicant: Adobe Inc.

    Abstract: A 3D fluid volume generation system obtains a 2D sketch of an outline of a fluid for which the 3D fluid volume is to be generated, and generates a 3D fluid volume that matches the user's sketch. The 3D fluid volume generation system implements a coarse volume generation stage followed by a refinement stage. In the coarse volume generation stage, the 3D fluid volume generation system generates a coarse 3D fluid volume based on the 2D sketch. The coarse 3D fluid volume is referred to as “coarse” because the contour of the coarse 3D fluid volume roughly matches the 2D sketch. In the refinement stage, the coarse 3D fluid volume is refined to better match the 2D sketch, and the 3D fluid volume for the 2D sketch is output.

    Forecasting multiple poses based on a graphical image

    公开(公告)号:US10475207B2

    公开(公告)日:2019-11-12

    申请号:US16057161

    申请日:2018-08-07

    Applicant: Adobe Inc.

    Abstract: A forecasting neural network receives data and extracts features from the data. A recurrent neural network included in the forecasting neural network provides forecasted features based on the extracted features. In an embodiment, the forecasting neural network receives an image, and features of the image are extracted. The recurrent neural network forecasts features based on the extracted features, and pose is forecasted based on the forecasted features. Additionally or alternatively, additional poses are forecasted based on additional forecasted features.

    Instance-level semantic segmentation system

    公开(公告)号:US10424064B2

    公开(公告)日:2019-09-24

    申请号:US15296845

    申请日:2016-10-18

    Applicant: Adobe Inc.

    Abstract: Certain aspects involve semantic segmentation of objects in a digital visual medium by determining a score for each pixel of the digital visual medium that is representative of a likelihood that each pixel corresponds to the objects associated with bounding boxes within the digital visual medium. An instance-level label that yields a label for each of the pixels of the digital visual medium corresponding to the objects is determined based, in part, on a collective probability map including the score for each pixel of the digital visual medium. In some aspects, the score for each pixel corresponding to each bounding box is determined by a prediction model trained by a neural network.

    OIL PAINTING STROKE SIMULATION USING NEURAL NETWORK

    公开(公告)号:US20190147627A1

    公开(公告)日:2019-05-16

    申请号:US15814751

    申请日:2017-11-16

    Applicant: Adobe Inc.

    Abstract: Oil painting simulation techniques are disclosed which simulate painting brush strokes using a trained neural network. In some examples, a method may include inferring a new height map of existing paint on a canvas after a new painting brush stroke is applied based on a bristle trajectory map that represents the new painting brush stroke and a height map of existing paint on the canvas prior to the application of the new painting brush stroke, and generating a rendering of the new painting brush stroke based on the new height map of existing paint on the canvas after the new painting brush stroke is applied to the canvas and a color map.

    Multi-style texture synthesis
    80.
    发明授权

    公开(公告)号:US10192321B2

    公开(公告)日:2019-01-29

    申请号:US15409321

    申请日:2017-01-18

    Applicant: ADOBE INC.

    Abstract: Systems and techniques that synthesize an image with similar texture to a selected style image. A generator network is trained to synthesize texture images depending on a selection unit input. The training configures the generator network to synthesize texture images that are similar to individual style images of multiple style images based on which is selected by the selection unit input. The generator network can be configured to minimize a covariance matrix-based style loss and/or a diversity loss in synthesizing the texture images. After training the generator network, the generator network is used to synthesize texture images for selected style images. For example, this can involve receiving user input selecting a selected style image, determining the selection unit input based on the selected style image, and synthesizing texture images using the generator network with the selection unit input and noise input.

Patent Agency Ranking