MULTI-SOURCE PANOPTIC FEATURE PYRAMID NETWORK

    公开(公告)号:US20230154185A1

    公开(公告)日:2023-05-18

    申请号:US17454740

    申请日:2021-11-12

    Applicant: ADOBE INC.

    Abstract: Systems and methods for image processing are described. Embodiments of the present disclosure receive an image having a plurality of object instances; encode the image to obtain image features; decode the image features to obtain object features; generate object detection information based on the object features using an object detection branch, wherein the object detection branch is trained based on a first training set using a detection loss; generate semantic segmentation information based on the object features using a semantic segmentation branch, wherein the semantic segmentation branch is trained based on a second training set different from the first training set using a semantic segmentation loss; and combine the object detection information and the semantic segmentation information to obtain panoptic segmentation information that indicates which pixels of the image correspond to each of the plurality of object instances.

    DETECTING DIGITAL OBJECTS AND GENERATING OBJECT MASKS ON DEVICE

    公开(公告)号:US20230128792A1

    公开(公告)日:2023-04-27

    申请号:US17589114

    申请日:2022-01-31

    Applicant: Adobe Inc.

    Abstract: The present disclosure relates to systems, methods, and non-transitory computer-readable media that generates object masks for digital objects portrayed in digital images utilizing a detection-masking neural network pipeline. In particular, in one or more embodiments, the disclosed systems utilize detection heads of a neural network to detect digital objects portrayed within a digital image. In some cases, each detection head is associated with one or more digital object classes that are not associated with the other detection heads. Further, in some cases, the detection heads implement multi-scale synchronized batch normalization to normalize feature maps across various feature levels. The disclosed systems further utilize a masking head of the neural network to generate one or more object masks for the detected digital objects. In some cases, the disclosed systems utilize post-processing techniques to filter out low-quality masks.

    Knowledge distillation for neural networks using multiple augmentation strategies

    公开(公告)号:US11610393B2

    公开(公告)日:2023-03-21

    申请号:US17062157

    申请日:2020-10-02

    Applicant: Adobe Inc.

    Abstract: The present disclosure relates to systems, methods, and non-transitory computer readable media for accurately and efficiently learning parameters of a distilled neural network from parameters of a source neural network utilizing multiple augmentation strategies. For example, the disclosed systems can generate lightly augmented digital images and heavily augmented digital images. The disclosed systems can further learn parameters for a source neural network from the lightly augmented digital images. Moreover, the disclosed systems can learn parameters for a distilled neural network from the parameters learned for the source neural network. For example, the disclosed systems can compare classifications of heavily augmented digital images generated by the source neural network and the distilled neural network to transfer learned parameters from the source neural network to the distilled neural network via a knowledge distillation loss function.

    Generating refined alpha mattes utilizing guidance masks and a progressive refinement network

    公开(公告)号:US11593948B2

    公开(公告)日:2023-02-28

    申请号:US17177595

    申请日:2021-02-17

    Applicant: Adobe Inc.

    Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize a progressive refinement network to refine alpha mattes generated utilizing a mask-guided matting neural network. In particular, the disclosed systems can use the matting neural network to process a digital image and a coarse guidance mask to generate alpha mattes at discrete neural network layers. In turn, the disclosed systems can use the progressive refinement network to combine alpha mattes and refine areas of uncertainty. For example, the progressive refinement network can combine a core alpha matte corresponding to more certain core regions of a first alpha matte and a boundary alpha matte corresponding to uncertain boundary regions of a second, higher resolution alpha matte. Based on the combination of the core alpha matte and the boundary alpha matte, the disclosed systems can generate a final alpha matte for use in image matting processes.

    MODIFYING DIGITAL IMAGES UTILIZING A LANGUAGE GUIDED IMAGE EDITING MODEL

    公开(公告)号:US20230042221A1

    公开(公告)日:2023-02-09

    申请号:US17384109

    申请日:2021-07-23

    Applicant: Adobe Inc.

    Inventor: Ning Xu Zhe Lin

    Abstract: This disclosure describes one or more implementations of systems, non-transitory computer-readable media, and methods that perform language guided digital image editing utilizing a cycle-augmentation generative-adversarial neural network (CAGAN) that is augmented using a cross-modal cyclic mechanism. For example, the disclosed systems generate an editing description network that generates language embeddings which represent image transformations applied between a digital image and a modified digital image. The disclosed systems can further train a GAN to generate modified images by providing an input image and natural language embeddings generated by the editing description network (representing various modifications to the digital image from a ground truth modified image). In some instances, the disclosed systems also utilize an image request attention approach with the GAN to generate images that include adaptive edits in different spatial locations of the image.

    Hierarchical multiclass exposure defects classification in images

    公开(公告)号:US11494886B2

    公开(公告)日:2022-11-08

    申请号:US16888473

    申请日:2020-05-29

    Applicant: ADOBE INC.

    Abstract: Embodiments of the present invention provide systems, methods, and computer storage media for detecting and classifying an exposure defect in an image using neural networks trained via a limited amount of labeled training images. An image may be applied to a first neural network to determine whether the images includes an exposure defect. Detected defective image may be applied to a second neural network to determine an exposure defect classification for the image. The exposure defect classification can includes severe underexposure, medium underexposure, mild underexposure, mild overexposure, medium overexposure, severe overexposure, and/or the like. The image may be presented to a user along with the exposure defect classification.

    DETERMINING COLORS OF OBJECTS IN DIGITAL IMAGES

    公开(公告)号:US20220237826A1

    公开(公告)日:2022-07-28

    申请号:US17658799

    申请日:2022-04-11

    Applicant: Adobe Inc.

    Abstract: The present disclosure relates to a color classification system that accurately classifies objects in digital images based on color. In particular, in one or more embodiments, the color classification system utilizes a multidimensional color space and one or more color mappings to match objects to colors. Indeed, the color classification system can accurately and efficiently detect the color of an object utilizing one or more color similarity regions generated in the multidimensional color space.

    ENHANCED IMAGE SEARCH VIA CONTROLLABLE ATTRIBUTES

    公开(公告)号:US20220164380A1

    公开(公告)日:2022-05-26

    申请号:US17104745

    申请日:2020-11-25

    Applicant: Adobe Inc.

    Abstract: A query image is received, along with a query to initiate a search process to find other images based on the query image. The query includes a preference value associated with an attribute, the preference value indicative of a level of emphasis to be placed on the attribute during the search. A full query vector, which is within a first dimensional space and representative of the query image, is generated. The full query vector is projected to a reduced dimensional space having a dimensionality lower than the first dimensional space, to generate a query vector. An attribute direction corresponding to the attribute is identified. A plurality of candidate vectors of the reduced dimensional space is searched, based on the attribute direction, the query vector, and the preference value, to identify a target vector of the plurality of candidate vectors. A target image, representative of the target vector, is displayed.

    IMAGE SEGMENTATION USING TEXT EMBEDDING

    公开(公告)号:US20220156992A1

    公开(公告)日:2022-05-19

    申请号:US16952008

    申请日:2020-11-18

    Applicant: Adobe Inc.

    Abstract: A non-transitory computer-readable medium includes program code that is stored thereon. The program code is executable by one or more processing devices for performing operations including generating, by a model that includes trainable components, a learned image representation of a target image. The operations further include generating, by a text embedding model, a text embedding of a text query. The text embedding and the learned image representation of the target image are in a same embedding space. Additionally, the operations include generating a class activation map of the target image by, at least, convolving the learned image representation of the target image with the text embedding of the text query. Moreover, the operations include generating an object-segmented image using the class activation map of the target image.

Patent Agency Ranking