公开(公告)号：US11663481B2

公开(公告)日：2023-05-30

申请号：US16799191

申请日：2020-02-24

Applicant: Adobe Inc.

Inventor： Shikun Liu ,  Zhe Lin ,  Yilin Wang ,  Jianming Zhang ,  Federico Perazzi

IPC: G06N3/082 ,  G06N3/04

CPC classification number: G06N3/082 ,  G06N3/04

Abstract: The disclosure describes one or more implementations of a neural network architecture pruning system that automatically and progressively prunes neural networks. For instance, the neural network architecture pruning system can automatically reduce the size of an untrained or previously-trained neural network without reducing the accuracy of the neural network. For example, the neural network architecture pruning system jointly trains portions of a neural network while progressively pruning redundant subsets of the neural network at each training iteration. In many instances, the neural network architecture pruning system increases the accuracy of the neural network by progressively removing excess or redundant portions (e.g., channels or layers) of the neural network. Further, by removing portions of a neural network, the neural network architecture pruning system can increase the efficiency of the neural network.

公开(公告)号：US11663264B2

公开(公告)日：2023-05-30

申请号：US16785410

申请日：2020-02-07

Applicant: Adobe Inc.

Inventor： Pramod Srinivasan ,  Zhe Lin ,  Samarth Gulati ,  Saeid Motiian ,  Midhun Harikumar ,  Baldo Antonio Faieta ,  Alex C. Filipkowski

IPC: G06F16/532 ,  G06F16/583 ,  G06F16/538 ,  G06F40/30 ,  G06F16/51 ,  G06F16/54

CPC classification number: G06F16/532 ,  G06F16/51 ,  G06F16/538 ,  G06F16/54 ,  G06F16/583 ,  G06F40/30

Abstract: Keyword localization digital image search techniques are described. These techniques support an ability to indicate “where” a corresponding keyword is to be expressed with respect to a layout in a respective digital image resulting from a search query. The search query may also include an indication of a size of the keyword as expressed in the digital image, a number of instances of the keyword, and so forth. Additionally, the techniques and systems as described herein support real time search through use of keyword signatures.

公开(公告)号：US20230133522A1

公开(公告)日：2023-05-04

申请号：US17513127

申请日：2021-10-28

Applicant: Adobe Inc.

Inventor： Handong Zhao ,  Zhe Lin ,  Zhaowen Wang ,  Zhankui He ,  Ajinkya Gorakhnath Kale

IPC: G06F16/245 ,  G06F16/248 ,  G06N20/00

Abstract: Digital content search techniques are described that overcome the challenges found in conventional sequence-based techniques through use of a query-aware sequential search. In one example, a search query is received and sequence input data is obtained based on the search query. The sequence input data describes a sequence of digital content and respective search queries. Embedding data is generated based on the sequence input data using an embedding module of a machine-learning model. The embedding module includes a query-aware embedding layer that generates embeddings of the sequence of digital content and respective search queries. A search result is generated referencing at least one item of digital content by processing the embedding data using at least one layer of the machine-learning model.

公开(公告)号：US20230116969A1

公开(公告)日：2023-04-20

申请号：US17501191

申请日：2021-10-14

Applicant: Adobe Inc.

Inventor： Handong Zhao ,  Zhankui He ,  Zhaowen Wang ,  Ajinkya Gorakhnath Kale ,  Zhe Lin

IPC: G06F16/438 ,  G06F16/44 ,  G06N3/04

Abstract: Digital content search techniques are described. In one example, the techniques are incorporated as part of a multi-head self-attention module of a transformer using machine learning. A localized self-attention module, for instance, is incorporated as part of the multi-head self-attention module that applies local constraints to the sequence. This is performable in a variety of ways. In a first instance, a model-based local encoder is used, examples of which include a fixed-depth recurrent neural network (RNN) and a convolutional network. In a second instance, a masking-based local encoder is used, examples of which include use of a fixed window, Gaussian initialization, and an adaptive predictor.

公开(公告)号：US11594077B2

公开(公告)日：2023-02-28

申请号：US17025477

申请日：2020-09-18

Applicant: Adobe Inc.

Inventor： Trung Bui ,  Zhe Lin ,  Walter Chang ,  Nham Le ,  Franck Dernoncourt

IPC: G06V40/20 ,  G06N3/04 ,  G10L15/26 ,  G10L15/25

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for generating modified digital images based on verbal and/or gesture input by utilizing a natural language processing neural network and one or more computer vision neural networks. The disclosed systems can receive verbal input together with gesture input. The disclosed systems can further utilize a natural language processing neural network to generate a verbal command based on verbal input. The disclosed systems can select a particular computer vision neural network based on the verbal input and/or the gesture input. The disclosed systems can apply the selected computer vision neural network to identify pixels within a digital image that correspond to an object indicated by the verbal input and/or gesture input. Utilizing the identified pixels, the disclosed systems can generate a modified digital image by performing one or more editing actions indicated by the verbal input and/or gesture input.

公开(公告)号：US11574142B2

公开(公告)日：2023-02-07

申请号：US16943511

申请日：2020-07-30

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xihui Liu ,  Quan Hung Tran ,  Jianming Zhang ,  Handong Zhao

IPC: G06K9/62 ,  G06T11/00 ,  G06V10/44

Abstract: The technology described herein is directed to a reinforcement learning based framework for training a natural media agent to learn a rendering policy without human supervision or labeled datasets. The reinforcement learning based framework feeds the natural media agent a training dataset to implicitly learn the rendering policy by exploring a canvas and minimizing a loss function. Once trained, the natural media agent can be applied to any reference image to generate a series (or sequence) of continuous-valued primitive graphic actions, e.g., sequence of painting strokes, that when rendered by a synthetic rendering environment on a canvas, reproduce an identical or transformed version of the reference image subject to limitations of an action space and the learned rendering policy.

公开(公告)号：US11507800B2

公开(公告)日：2022-11-22

申请号：US15913829

申请日：2018-03-06

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Yufei Wang ,  Xiaohui Shen ,  Scott David Cohen ,  Jianming Zhang

IPC: G06T7/10 ,  G06F16/583 ,  G06N3/04 ,  G06N20/00

Abstract: Semantic segmentation techniques and systems are described that overcome the challenges of limited availability of training data to describe the potentially millions of tags that may be used to describe semantic classes in digital images. In one example, the techniques are configured to train neural networks to leverage different types of training datasets using sequential neural networks and use of vector representations to represent the different semantic classes.

公开(公告)号：US20220343108A1

公开(公告)日：2022-10-27

申请号：US17240246

申请日：2021-04-26

Applicant: ADOBE INC.

Inventor： Shipali Shetty ,  Zhe Lin ,  Alexander Smith

IPC: G06K9/62 ,  G06K9/68 ,  G06T7/00

Abstract: Systems and methods for image tagging are described. In some embodiments, images with problematic tags are identified after applying an auto-tagger. The images with problematic tags are then sent to an object detection network. In some cases, the object detection network is trained using a training set selected to improve detection of objects associated with the problematic tags. The output of the object detection network can be merged with the output of the auto-tagger to provide a combined image tagging output. In some cases, the output of the object detection network also includes a bounding box, which can be used to crop the image around a relevant object so that the auto-tagger can be reapplied to a portion of the image.

公开(公告)号：US11468110B2

公开(公告)日：2022-10-11

申请号：US16800415

申请日：2020-02-25

Applicant: Adobe Inc.

Inventor： Walter Wei Tuh Chang ,  Khoi Pham ,  Scott Cohen ,  Zhe Lin ,  Zhihong Ding

IPC: G06F16/532 ,  G06F16/583 ,  G06F16/538 ,  G06F16/33 ,  G06T11/60 ,  G06K9/62 ,  G06F40/279 ,  G06F40/247 ,  G06N20/00 ,  G06F16/242 ,  G06F16/28 ,  G06F40/30

Abstract: The present disclosure relates to an object selection system that automatically detects and selects objects in a digital image based on natural language-based inputs. For instance, the object selection system can utilize natural language processing tools to detect objects and their corresponding relationships within natural language object selection queries. For example, the object selection system can determine alternative object terms for unrecognized objects in a natural language object selection query. As another example, the object selection system can determine multiple types of relationships between objects in a natural language object selection query and utilize different object relationship models to select the requested query object.

公开(公告)号：US20220292650A1

公开(公告)日：2022-09-15

申请号：US17202019

申请日：2021-03-15

Applicant: Adobe Inc.

Inventor： Sohrab Amirghodsi ,  Lingzhi Zhang ,  Zhe Lin ,  Connelly Barnes ,  Elya Shechtman

IPC: G06T5/00 ,  G06T7/50 ,  G06T7/11 ,  G06T3/40 ,  G06N3/08

Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately, efficiently, and flexibly generating modified digital images utilizing a guided inpainting approach that implements a patch match model informed by a deep visual guide. In particular, the disclosed systems can utilize a visual guide algorithm to automatically generate guidance maps to help identify replacement pixels for inpainting regions of digital images utilizing a patch match model. For example, the disclosed systems can generate guidance maps in the form of structure maps, depth maps, or segmentation maps that respectively indicate the structure, depth, or segmentation of different portions of digital images. Additionally, the disclosed systems can implement a patch match model to identify replacement pixels for filling regions of digital images according to the structure, depth, and/or segmentation of the digital images.