公开(公告)号：US10453491B2

公开(公告)日：2019-10-22

申请号：US16273981

申请日：2019-02-12

Applicant: Adobe Inc.

Inventor： Geoffrey Oxholm ,  Elya Shechtman ,  Oliver Wang

IPC: G11B27/00 ,  H04N7/01 ,  G06K9/00 ,  G06K9/46

Abstract: Provided are video processing architectures and techniques configured to generate looping video. The video processing architectures and techniques automatically produce a looping video from a fixed-length video clip. Embodiments of the video processing architectures and techniques determine a lower-resolution version of the fixed-length video clip, and detect a presence of edges within image frames in the lower-resolution version. A pair of image frames having similar edges is identified as a pair of candidates for a transition point (i.e., a start frame and an end frame) at which the looping video can repeat. Using start and end frames having similar edges mitigates teleporting of objects displayed in the looping video. In some cases, teleporting during repeating is eliminated.

公开(公告)号：US20190306451A1

公开(公告)日：2019-10-03

申请号：US15937349

申请日：2018-03-27

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  Pedro Morgado ,  Timothy Langlois

IPC: H04N5/60 ,  H04N5/04

Abstract: Certain embodiments involve generating and providing spatial audio using a predictive model. For example, a generates, using a predictive model, a visual representation of visual content provideable to a user device by encoding the visual content into the visual representation that indicates a visual element in the visual content. The system generates, using the predictive model, an audio representation of audio associated with the visual content by encoding the audio into the audio representation that indicates an audio element in the audio. The system also generates, using the predictive model, spatial audio based at least in part on the audio element and associating the spatial audio with the visual element. The system can also augment the visual content using the spatial audio by at least associating the spatial audio with the visual content.

公开(公告)号：US20240320789A1

公开(公告)日：2024-09-26

申请号：US18585957

申请日：2024-02-23

Applicant: ADOBE INC.

Inventor： Tobias Hinz ,  Taesung Park ,  Jingwan Lu ,  Elya Shechtman ,  Richard Zhang ,  Oliver Wang

IPC: G06T3/4053 ,  G06T3/4046 ,  G06T11/00

CPC classification number: G06T3/4053 ,  G06T3/4046 ,  G06T11/00

Abstract: A method, non-transitory computer readable medium, apparatus, and system for image generation include obtaining an input image having a first resolution, where the input image includes random noise, and generating a low-resolution image based on the input image, where the low-resolution image has the first resolution. The method, non-transitory computer readable medium, apparatus, and system further include generating a high-resolution image based on the low-resolution image, where the high-resolution image has a second resolution that is greater than the first resolution.

公开(公告)号：US12039657B2

公开(公告)日：2024-07-16

申请号：US17204571

申请日：2021-03-17

Applicant: ADOBE INC.

Inventor： Oliver Wang ,  Simon Niklaus ,  Zhengqi Li

IPC: G06T15/04 ,  G06T7/557 ,  G06T7/90 ,  G06T15/50

CPC classification number: G06T15/04 ,  G06T7/557 ,  G06T7/90 ,  G06T15/506 ,  G06T2207/10016 ,  G06T2207/20084 ,  G06T2207/30244

Abstract: Embodiments of the technology described herein, provide a view and time synthesis of dynamic scenes captured by a camera. The technology described herein represents a dynamic scene as a continuous function of both space and time. The technology may parameterize this function with a deep neural network (a multi-layer perceptron (MLP)), and perform rendering using volume tracing. At a very high level, a dynamic scene depicted in the video may be used to train the MLP. Once trained, the MLP is able to synthesize a view of the scene at a time and/or camera pose not found in the video through prediction. As used herein, a dynamic scene comprises one or more moving objects.

公开(公告)号：US11908036B2

公开(公告)日：2024-02-20

申请号：US17034467

申请日：2020-09-28

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  Jianming Zhang ,  Dingzeyu Li ,  Zekun Hao

IPC: G06T1/00 ,  G06N3/08 ,  G06T7/11 ,  G06T7/55 ,  G06V10/44

CPC classification number: G06T1/0014 ,  G06N3/08 ,  G06T1/0007 ,  G06T7/11 ,  G06T7/55 ,  G06V10/44 ,  G06T2207/20081

Abstract: The technology described herein is directed to a cross-domain training framework that iteratively trains a domain adaptive refinement agent to refine low quality real-world image acquisition data, e.g., depth maps, when accompanied by corresponding conditional data from other modalities, such as the underlying images or video from which the image acquisition data is computed. The cross-domain training framework includes a shared cross-domain encoder and two conditional decoder branch networks, e.g., a synthetic conditional depth prediction branch network and a real conditional depth prediction branch network. The shared cross-domain encoder converts synthetic and real-world image acquisition data into synthetic and real compact feature representations, respectively. The synthetic and real conditional decoder branch networks convert the respective synthetic and real compact feature representations back to synthetic and real image acquisition data (refined versions) conditioned on data from the other modalities. The cross-domain training framework iteratively trains the domain adaptive refinement agent.

公开(公告)号：US20240046430A1

公开(公告)日：2024-02-08

申请号：US18375187

申请日：2023-09-29

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  John Nelson ,  Geoffrey Oxholm ,  Elya Shechtman

IPC: G06T5/00 ,  G06T7/269

CPC classification number: G06T5/005 ,  G06T7/269 ,  G06T2207/10016

Abstract: One or more processing devices access a scene depicting a reference object that includes an annotation identifying a target region to be modified in one or more video frames. The one or more processing devices determine that a target pixel corresponds to a sub-region within the target region that includes hallucinated content. The one or more processing devices determine gradient constraints using gradient values of neighboring pixels in the hallucinated content, the neighboring pixels being adjacent to the target pixel and corresponding to four cardinal directions. The one or more processing devices update color data of the target pixel subject to the determined gradient constraints.

公开(公告)号：US11880766B2

公开(公告)日：2024-01-23

申请号：US17384357

申请日：2021-07-23

Applicant: Adobe Inc.

Inventor： Cameron Smith ,  Ratheesh Kalarot ,  Wei-An Lin ,  Richard Zhang ,  Niloy Mitra ,  Elya Shechtman ,  Shabnam Ghadar ,  Zhixin Shu ,  Yannick Hold-Geoffrey ,  Nathan Carr ,  Jingwan Lu ,  Oliver Wang ,  Jun-Yan Zhu

IPC: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06T11/60 ,  G06T3/40 ,  G06N20/20 ,  G06T5/00 ,  G06T5/20 ,  G06T3/00 ,  G06T11/00 ,  G06F18/40 ,  G06F18/211 ,  G06F18/214 ,  G06F18/21 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06F3/04845 ,  G06F3/04847 ,  G06F18/211 ,  G06F18/214 ,  G06F18/2163 ,  G06F18/40 ,  G06N3/045 ,  G06N20/20 ,  G06T3/0006 ,  G06T3/0093 ,  G06T3/40 ,  G06T3/4038 ,  G06T3/4046 ,  G06T5/005 ,  G06T5/20 ,  G06T11/001 ,  G06T11/60 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20221 ,  G06T2210/22

Abstract: An improved system architecture uses a pipeline including a Generative Adversarial Network (GAN) including a generator neural network and a discriminator neural network to generate an image. An input image in a first domain and information about a target domain are obtained. The domains correspond to image styles. An initial latent space representation of the input image is produced by encoding the input image. An initial output image is generated by processing the initial latent space representation with the generator neural network. Using the discriminator neural network, a score is computed indicating whether the initial output image is in the target domain. A loss is computed based on the computed score. The loss is minimized to compute an updated latent space representation. The updated latent space representation is processed with the generator neural network to generate an output image in the target domain.

公开(公告)号：US11871145B2

公开(公告)日：2024-01-09

申请号：US17223945

申请日：2021-04-06

Applicant: Adobe Inc.

Inventor： Simon Niklaus ,  Oliver Wang ,  Long Mai

IPC: H04N7/01 ,  G06N3/08 ,  G06T5/20 ,  G06T5/50 ,  G06T5/00 ,  G06N3/04

CPC classification number: H04N7/0135 ,  G06N3/04 ,  G06N3/08 ,  G06T5/002 ,  G06T5/003 ,  G06T5/20 ,  G06T5/50 ,  G06T2207/10016 ,  G06T2207/20004 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20212

Abstract: Embodiments are disclosed for video image interpolation. In some embodiments, video image interpolation includes receiving a pair of input images from a digital video, determining, using a neural network, a plurality of spatially varying kernels each corresponding to a pixel of an output image, convolving a first set of spatially varying kernels with a first input image from the pair of input images and a second set of spatially varying kernels with a second input image from the pair of input images to generate filtered images, and generating the output image by performing kernel normalization on the filtered images.

公开(公告)号：US11823357B2

公开(公告)日：2023-11-21

申请号：US17196581

申请日：2021-03-09

Applicant: Adobe Inc.

Inventor： Oliver Wang ,  John Nelson ,  Geoffrey Oxholm ,  Elya Shechtman

IPC: G06T5/00 ,  G06T7/269

CPC classification number: G06T5/005 ,  G06T7/269 ,  G06T2207/10016

Abstract: Certain aspects involve video inpainting in which content is propagated from a user-provided reference video frame to other video frames depicting a scene. One example method includes one or more processing devices that performs operations that include accessing a scene depicting a reference object that includes an annotation identifying a target region to be modified in one or more video frames. The operations also includes computing a target motion of a target pixel that is subject to a motion constraint. The motion constraint is based on a three-dimensional model of the reference object. Further, operations include determining color data of the target pixel to correspond to the target motion. The color data includes a color value and a gradient. Operations also include determining gradient constraints using gradient values of neighbor pixels. Additionally, the processing devices updates the color data of the target pixel subject to the gradient constraints.

公开(公告)号：US11544880B2

公开(公告)日：2023-01-03

申请号：US16874399

申请日：2020-05-14

Applicant: Adobe Inc.

Inventor： Taesung Park ,  Richard Zhang ,  Oliver Wang ,  Junyan Zhu ,  Jingwan Lu ,  Elya Shechtman ,  Alexei A Efros

IPC: G06T9/00 ,  G06N3/08 ,  G06T3/40

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating a modified digital image from extracted spatial and global codes. For example, the disclosed systems can utilize a global and spatial autoencoder to extract spatial codes and global codes from digital images. The disclosed systems can further utilize the global and spatial autoencoder to generate a modified digital image by combining extracted spatial and global codes in various ways for various applications such as style swapping, style blending, and attribute editing.