公开(公告)号：US11232547B2

公开(公告)日：2022-01-25

申请号：US16930736

申请日：2020-07-16

Applicant: Adobe Inc.

Inventor： Chen Fang ,  Zhe Lin ,  Zhaowen Wang ,  Yulun Zhang ,  Yilin Wang ,  Jimei Yang

IPC: G06T11/00 ,  G06T5/50 ,  G06T3/40 ,  G06T7/11 ,  G06T3/00

Abstract: A style of a digital image is transferred to another digital image of arbitrary resolution. A high-resolution (HR) content image is segmented into several low-resolution (LR) patches. The resolution of a style image is matched to have the same resolution as the LR content image patches. Style transfer is then performed on a patch-by-patch basis using, for example, a pair of feature transforms—whitening and coloring. The patch-by-patch style transfer process is then repeated at several increasing resolutions, or scale levels, of both the content and style images. The results of the style transfer at each scale level are incorporated into successive scale levels up to and including the original HR scale. As a result, style transfer can be performed with images having arbitrary resolutions to produce visually pleasing results with good spatial consistency.

公开(公告)号：US20200226724A1

公开(公告)日：2020-07-16

申请号：US16246051

申请日：2019-01-11

Applicant: Adobe Inc.

Inventor： Chen Fang ,  Zhe Lin ,  Zhaowen Wang ,  Yulun Zhang ,  Yilin Wang ,  Jimei Yang

IPC: G06T5/00 ,  G06T9/00 ,  G06K9/62

Abstract: In implementations of transferring image style to content of a digital image, an image editing system includes an encoder that extracts features from a content image and features from a style image. A whitening and color transform generates coarse features from the content and style features extracted by the encoder for one pass of encoding and decoding. Hence, the processing delay and memory requirements are low. A feature transfer module iteratively transfers style features to the coarse feature map and generates a fine feature map. The image editing system fuses the fine features with the coarse features, and a decoder generates an output image with content of the content image in a style of the style image from the fused features. Accordingly, the image editing system efficiently transfers an image style to image content in real-time, without undesirable artifacts in the output image.

公开(公告)号：US11288771B2

公开(公告)日：2022-03-29

申请号：US16861688

申请日：2020-04-29

Applicant: ADOBE INC.

Inventor： Yulun Zhang ,  Zhifei Zhang ,  Jose Ignacio Echevarria Vallespi ,  Zhaowen Wang ,  Stephen Diverdi

IPC: G06T3/40

Abstract: Systems and methods for texture hallucination with a large upscaling factor are described. Embodiments of the systems and methods may receive an input image and a reference image, extract an upscaled feature map from the input image, match the input image to a portion of the reference image, wherein a resolution of the reference image is higher than a resolution of the input image, concatenate the upscaled feature map with a reference feature map corresponding to the portion of the reference image to produce a concatenated feature map, and generate a reconstructed image based on the concatenated feature map using a machine learning model trained with a texture loss and a degradation loss, wherein the texture loss is based on a high frequency band filter, and the degradation loss is based on a downscaled version of the reconstructed image.

公开(公告)号：US20210342974A1

公开(公告)日：2021-11-04

申请号：US16861688

申请日：2020-04-29

Applicant: ADOBE INC.

Inventor： Yulun Zhang ,  Zhifei Zhang ,  Jose Ignacio Echevarria Vallespi ,  Zhaowen Wang ,  Stephen Diverdi

IPC: G06T3/40 ,  G06N20/00

Abstract: Systems and methods for texture hallucination with a large upscaling factor are described. Embodiments of the systems and methods may receive an input image and a reference image, extract an upscaled feature map from the input image, match the input image to a portion of the reference image, wherein a resolution of the reference image is higher than a resolution of the input image, concatenate the upscaled feature map with a reference feature map corresponding to the portion of the reference image to produce a concatenated feature map, and generate a reconstructed image based on the concatenated feature map using a machine learning model trained with a texture loss and a degradation loss, wherein the texture loss is based on a high frequency band filter, and the degradation loss is based on a downscaled version of the reconstructed image.

公开(公告)号：US10769764B2

公开(公告)日：2020-09-08

申请号：US16271058

申请日：2019-02-08

Applicant: Adobe Inc.

Inventor： Chen Fang ,  Zhe Lin ,  Zhaowen Wang ,  Yulun Zhang ,  Yilin Wang ,  Jimei Yang

IPC: G06T7/40 ,  G06T5/50 ,  G06T3/40 ,  G06T7/11 ,  G06T11/00 ,  G06T3/00

Abstract: A style of a digital image is transferred to another digital image of arbitrary resolution. A high-resolution (HR) content image is segmented into several low-resolution (LR) patches. The resolution of a style image is matched to have the same resolution as the LR content image patches. Style transfer is then performed on a patch-by-patch basis using, for example, a pair of feature transforms—whitening and coloring. The patch-by-patch style transfer process is then repeated at several increasing resolutions, or scale levels, of both the content and style images. The results of the style transfer at each scale level are incorporated into successive scale levels up to and including the original HR scale. As a result, style transfer can be performed with images having arbitrary resolutions to produce visually pleasing results with good spatial consistency.

公开(公告)号：US20200349688A1

公开(公告)日：2020-11-05

申请号：US16930736

申请日：2020-07-16

Applicant: Adobe Inc.

Inventor： Chen Fang ,  Zhe Lin ,  Zhaowen Wang ,  Yulun Zhang ,  Yilin Wang ,  Jimei Yang

IPC: G06T5/50 ,  G06T3/40 ,  G06T11/00 ,  G06T3/00 ,  G06T7/11

Abstract: A style of a digital image is transferred to another digital image of arbitrary resolution. A high-resolution (HR) content image is segmented into several low-resolution (LR) patches. The resolution of a style image is matched to have the same resolution as the LR content image patches. Style transfer is then performed on a patch-by-patch basis using, for example, a pair of feature transforms—whitening and coloring. The patch-by-patch style transfer process is then repeated at several increasing resolutions, or scale levels, of both the content and style images. The results of the style transfer at each scale level are incorporated into successive scale levels up to and including the original HR scale. As a result, style transfer can be performed with images having arbitrary resolutions to produce visually pleasing results with good spatial consistency.

公开(公告)号：US20200258204A1

公开(公告)日：2020-08-13

申请号：US16271058

申请日：2019-02-08

Applicant: Adobe Inc.

Inventor： Chen Fang ,  Zhe Lin ,  Zhaowen Wang ,  Yulun Zhang ,  Yilin Wang ,  Jimei Yang

IPC: G06T5/50 ,  G06T3/40 ,  G06T7/11 ,  G06T3/00 ,  G06T11/00

Abstract: A style of a digital image is transferred to another digital image of arbitrary resolution. A high-resolution (HR) content image is segmented into several low-resolution (LR) patches. The resolution of a style image is matched to have the same resolution as the LR content image patches. Style transfer is then performed on a patch-by-patch basis using, for example, a pair of feature transforms—whitening and coloring. The patch-by-patch style transfer process is then repeated at several increasing resolutions, or scale levels, of both the content and style images. The results of the style transfer at each scale level are incorporated into successive scale levels up to and including the original HR scale. As a result, style transfer can be performed with images having arbitrary resolutions to produce visually pleasing results with good spatial consistency.