公开(公告)号：US11107219B2

公开(公告)日：2021-08-31

申请号：US16518880

申请日：2019-07-22

Applicant: Adobe Inc.

Inventor： Scott Cohen ,  Zhe Lin ,  Mingyang Ling

IPC: G06T7/00 ,  G06T7/11 ,  G06T7/90 ,  G06F16/33 ,  G06K9/46 ,  G06K9/62 ,  G10L15/22 ,  G06N3/04

Abstract: The present disclosure relates to an object selection system that accurately detects and automatically selects user-requested objects (e.g., query objects) in a digital image. For example, the object selection system builds and utilizes an object selection pipeline to determine which object detection neural network to utilize to detect a query object based on analyzing the object class of the query object. In addition, the object selection system can add, update, or replace portions of the object selection pipeline to improve overall accuracy and efficiency of automatic object selection within an image.

公开(公告)号：US20210264278A1

公开(公告)日：2021-08-26

申请号：US16799191

申请日：2020-02-24

Applicant: Adobe Inc.

Inventor： Shikun Liu ,  Zhe Lin ,  Yilin Wang ,  Jianming Zhang ,  Federico Perazzi

IPC: G06N3/08 ,  G06N3/04

Abstract: The disclosure describes one or more implementations of a neural network architecture pruning system that automatically and progressively prunes neural networks. For instance, the neural network architecture pruning system can automatically reduce the size of an untrained or previously-trained neural network without reducing the accuracy of the neural network. For example, the neural network architecture pruning system jointly trains portions of a neural network while progressively pruning redundant subsets of the neural network at each training iteration. In many instances, the neural network architecture pruning system increases the accuracy of the neural network by progressively removing excess or redundant portions (e.g., channels or layers) of the neural network. Further, by removing portions of a neural network, the neural network architecture pruning system can increase the efficiency of the neural network.

公开(公告)号：US20210232770A1

公开(公告)日：2021-07-29

申请号：US16775697

申请日：2020-01-29

Applicant: ADOBE INC.

Inventor： Zhe Lin ,  Walter W. Chang ,  Scott Cohen ,  Khoi Viet Pham ,  Jonathan Brandt ,  Franck Dernoncourt

IPC: G06F40/30 ,  G06F16/532 ,  G06F16/55 ,  G06N5/02 ,  G06N20/00 ,  G06N5/04 ,  G06K9/46 ,  G06F40/205 ,  G06F40/295

Abstract: Embodiments of the present invention provide systems, methods, and non-transitory computer storage media for parsing a given input referring expression into a parse structure and generating a semantic computation graph to identify semantic relationships among and between objects. At a high level, when embodiments of the preset invention receive a referring expression, a parse tree is created and mapped into a hierarchical subject, predicate, object graph structure that labeled noun objects in the referring expression, the attributes of the labeled noun objects, and predicate relationships (e.g., verb actions or spatial propositions) between the labeled objects. Embodiments of the present invention then transform the subject, predicate, object graph structure into a semantic computation graph that may be recursively traversed and interpreted to determine how noun objects, their attributes and modifiers, and interrelationships are provided to downstream image editing, searching, or caption indexing tasks.

公开(公告)号：US11042798B2

公开(公告)日：2021-06-22

申请号：US15082877

申请日：2016-03-28

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Jianchao Yang ,  Hailin Jin ,  Chen Fang

IPC: G06N3/08 ,  G06N5/04 ,  G06N20/00 ,  G06F16/532 ,  G06F16/58 ,  G06N3/04

Abstract: Certain embodiments involve learning features of content items (e.g., images) based on web data and user behavior data. For example, a system determines latent factors from the content items based on data including a user's text query or keyword query for a content item and the user's interaction with the content items based on the query (e.g., a user's click on a content item resulting from a search using the text query). The system uses the latent factors to learn features of the content items. The system uses a previously learned feature of the content items for iterating the process of learning features of the content items to learn additional features of the content items, which improves the accuracy with which the system is used to learn other features of the content items.

公开(公告)号：US20210082124A1

公开(公告)日：2021-03-18

申请号：US17103119

申请日：2020-11-24

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Wei Xiong ,  Connelly Barnes ,  Jimei Yang ,  Xin Lu

IPC: G06T7/149 ,  G06T7/13 ,  G06T7/194 ,  G06T5/00

Abstract: In some embodiments, an image manipulation application receives an incomplete image that includes a hole area lacking image content. The image manipulation application applies a contour detection operation to the incomplete image to detect an incomplete contour of a foreground object in the incomplete image. The hole area prevents the contour detection operation from detecting a completed contour of the foreground object. The image manipulation application further applies a contour completion model to the incomplete contour and the incomplete image to generate the completed contour for the foreground object. Based on the completed contour and the incomplete image, the image manipulation application generates image content for the hole area to generate a completed image.

公开(公告)号：US20210027497A1

公开(公告)日：2021-01-28

申请号：US16518795

申请日：2019-07-22

Applicant: Adobe Inc.

Inventor： Zhihong Ding ,  Scott Cohen ,  Zhe Lin ,  Mingyang Ling

IPC: G06T7/90 ,  G06K9/32 ,  G06K9/62 ,  G06N3/02

Abstract: The present disclosure relates to a color classification system that accurately classifies objects in digital images based on color. In particular, in one or more embodiments, the color classification system utilizes a multidimensional color space and one or more color mappings to match objects to colors. Indeed, the color classification system can accurately and efficiently detect the color of an object utilizing one or more color similarity regions generated in the multidimensional color space.

公开(公告)号：US20200380298A1

公开(公告)日：2020-12-03

申请号：US16426264

申请日：2019-05-30

Applicant: Adobe Inc.

Inventor： Pranav Vineet Aggarwal ,  Zhe Lin ,  Baldo Antonio Faieta ,  Saeid Motiian

IPC: G06K9/62 ,  G06K9/72 ,  G06F16/535 ,  G06N20/00

Abstract: Text-to-visual machine learning embedding techniques are described that overcome the challenges of conventional techniques in a variety of ways. These techniques include use of query-based training data which may expand availability and types of training data usable to train a model. Generation of negative digital image samples is also described that may increase accuracy in training the model using machine learning. A loss function is also described that also supports increased accuracy and computational efficiency by losses separately, e.g., between positive or negative sample embeddings a text embedding.

公开(公告)号：US20200342576A1

公开(公告)日：2020-10-29

申请号：US16928340

申请日：2020-07-14

Applicant: Adobe Inc.

Inventor： Zhe Lin ,  Xin Lu ,  Xiaohui Shen ,  Jimei Yang ,  Jiahui Yu

IPC: G06T5/00 ,  G06K9/62

Abstract: Digital image completion by learning generation and patch matching jointly is described. Initially, a digital image having at least one hole is received. This holey digital image is provided as input to an image completer formed with a dual-stage framework that combines a coarse image neural network and an image refinement network. The coarse image neural network generates a coarse prediction of imagery for filling the holes of the holey digital image. The image refinement network receives the coarse prediction as input, refines the coarse prediction, and outputs a filled digital image having refined imagery that fills these holes. The image refinement network generates refined imagery using a patch matching technique, which includes leveraging information corresponding to patches of known pixels for filtering patches generated based on the coarse prediction. Based on this, the image completer outputs the filled digital image with the refined imagery.

公开(公告)号：US10789525B2

公开(公告)日：2020-09-29

申请号：US15002172

申请日：2016-01-20

Applicant: ADOBE INC.

Inventor： Bernard James Kerr ,  Zhe Lin ,  Patrick Reynolds ,  Baldo Faieta

IPC: G06F17/00 ,  G06F7/00 ,  G06N3/02 ,  G06F16/583 ,  G06F3/0484

Abstract: In various implementations, one or more specific attributes found in an image can be modified utilizing one or more specific attributes found in another image. Machine learning, deep neural networks, and other computer vision techniques can be utilized to extract attributes of images, such as color, composition, font, style, and texture from one or more images. A user may modify at least one of these attributes in a first image based on the attribute(s) of another image and initiate a visual-based search using the modified image.

公开(公告)号：US10783622B2

公开(公告)日：2020-09-22

申请号：US15962735

申请日：2018-04-25

Applicant: Adobe Inc.

Inventor： Yilin Wang ,  Zhe Lin ,  Zhaowen Wang ,  Xin Lu ,  Xiaohui Shen ,  Chih-Yao Hsieh

IPC: G06K9/00 ,  G06K9/62 ,  G03B7/00 ,  G06T5/50

Abstract: The present disclosure relates to training and utilizing an image exposure transformation network to generate a long-exposure image from a single short-exposure image (e.g., still image). In various embodiments, the image exposure transformation network is trained using adversarial learning, long-exposure ground truth images, and a multi-term loss function. In some embodiments, the image exposure transformation network includes an optical flow prediction network and/or an appearance guided attention network. Trained embodiments of the image exposure transformation network generate realistic long-exposure images from single short-exposure images without additional information.