CONVOLUTIONAL NEURAL NETWORKS WITH ADJUSTABLE FEATURE RESOLUTIONS AT RUNTIME

    公开(公告)号:US20210232927A1

    公开(公告)日:2021-07-29

    申请号:US16751897

    申请日:2020-01-24

    Applicant: Adobe Inc.

    Abstract: In some embodiments, an application receives a request to execute a convolutional neural network model. The application determines the computational complexity requirement for the neural network based on the computing resource available on the device. The application further determines the architecture of the convolutional neural network model by determining the locations of down-sampling layers within the convolutional neural network model based on the computational complexity requirement. The application reconfigures the architecture of the convolutional neural network model by moving the down-sampling layers to the determined locations and executes the convolutional neural network model to generate output results.

    Aesthetics-guided image enhancement

    公开(公告)号:US11069030B2

    公开(公告)日:2021-07-20

    申请号:US15928706

    申请日:2018-03-22

    Applicant: ADOBE INC.

    Abstract: Methods and systems are provided for generating enhanced image. A neural network system is trained where the training includes training a first neural network that generates enhanced images conditioned on content of an image undergoing enhancement and training a second neural network that designates realism of the enhanced images generated by the first neural network. The neural network system is trained by determine loss and accordingly adjusting the appropriate neural network(s). The trained neural network system is used to generate an enhanced aesthetic image from a selected image where the output enhanced aesthetic image has increased aesthetics when compared to the selected image.

    Utilizing a large-scale object detector to automatically select objects in digital images

    公开(公告)号:US11055566B1

    公开(公告)日:2021-07-06

    申请号:US16817418

    申请日:2020-03-12

    Applicant: Adobe Inc.

    Abstract: The present disclosure relates to an object selection system that automatically detects and selects objects in a digital image utilizing a large-scale object detector. For instance, in response to receiving a request to automatically select a query object with an unknown object class in a digital image, the object selection system can utilize a large-scale object detector to detect potential objects in the image, filter out one or more potential objects, and label the remaining potential objects in the image to detect the query object. In some implementations, the large-scale object detector utilizes a region proposal model, a concept mask model, and an auto tagging model to automatically detect objects in the digital image.

    Generating neutral-pose transformations of self-portrait images

    公开(公告)号:US11024060B1

    公开(公告)日:2021-06-01

    申请号:US16812669

    申请日:2020-03-09

    Applicant: Adobe Inc.

    Abstract: Techniques are provided for converting a self-portrait image into a neutral-pose portrait image, including receiving a self-portrait input image, which contains at least one person who is the subject of the self-portrait. A nearest pose search selects a target neutral-pose image that closely matches or approximates the pose of the upper torso region of the subject in the self-portrait input image. Coordinate-based inpainting maps pixels from the upper torso region in the self-portrait input image to corresponding regions in the selected target neutral-pose image to produce a coarse result image. A neutral-pose composition refines the coarse result image by synthesizing details in the body region of the subject (which in some cases includes the subject's head, arms, and torso), and inpainting pixels into missing portions of the background. The refined image is composited with the original self-portrait input image to produce a neutral-pose result image.

    Frame selection based on a trained neural network

    公开(公告)号:US10990877B2

    公开(公告)日:2021-04-27

    申请号:US15866129

    申请日:2018-01-09

    Applicant: Adobe Inc.

    Abstract: Various embodiments describe frame selection based on training and using a neural network. In an example, the neural network is a convolutional neural network trained with training pairs. Each training pair includes two training frames from a frame collection. The loss function relies on the estimated quality difference between the two training frames. Further, the definition of the loss function varies based on the actual quality difference between these two frames. In a further example, the neural network is trained by incorporating facial heatmaps generated from the training frames and facial quality scores of faces detected in the training frames. In addition, the training involves using a feature mean that represents an average of the features of the training frames belonging to the same frame collection. Once the neural network is trained, a frame collection is input thereto and a frame is selected based on generated quality scores.

    Learning copy space using regression and segmentation neural networks

    公开(公告)号:US10970599B2

    公开(公告)日:2021-04-06

    申请号:US16191724

    申请日:2018-11-15

    Applicant: ADOBE INC.

    Abstract: Techniques are disclosed for characterizing and defining the location of a copy space in an image. A methodology implementing the techniques according to an embodiment includes applying a regression convolutional neural network (CNN) to an image. The regression CNN is configured to predict properties of the copy space such as size and type (natural or manufactured). The prediction is conditioned on a determination of the presence of the copy space in the image. The method further includes applying a segmentation CNN to the image. The segmentation CNN is configured to generate one or more pixel-level masks to define the location of copy spaces in the image, whether natural or manufactured, or to define the location of a background region of the image. The segmentation CNN may include a first stage comprising convolutional layers and a second stage comprising pairs of boundary refinement layers and bilinear up-sampling layers.

    Recurrent neural network architectures which provide text describing images

    公开(公告)号:US10949744B2

    公开(公告)日:2021-03-16

    申请号:US16507960

    申请日:2019-07-10

    Applicant: Adobe Inc.

    Abstract: Provided are systems and techniques that provide an output phrase describing an image. An example method includes creating, with a convolutional neural network, feature maps describing image features in locations in the image. The method also includes providing a skeletal phrase for the image by processing the feature maps with a first long short-term memory (LSTM) neural network trained based on a first set of ground truth phrases which exclude attribute words. Then, attribute words are provided by processing the skeletal phrase and the feature maps with a second LSTM neural network trained based on a second set of ground truth phrases including words for attributes. Then, the method combines the skeletal phrase and the attribute words to form the output phrase.

    COMPRESSION OF MACHINE LEARNING MODELS

    公开(公告)号:US20210073644A1

    公开(公告)日:2021-03-11

    申请号:US16563226

    申请日:2019-09-06

    Applicant: Adobe Inc.

    Abstract: A machine learning model compression system and related techniques are described herein. The machine learning model compression system can intelligently remove certain parameters of a machine learning model, without introducing a loss in performance of the machine learning model. Various parameters of a machine learning model can be removed during compression of the machine learning model, such as one or more channels of a single-branch or multi-branch neural network, one or more branches of a multi-branch neural network, certain weights of a channel of a single-branch or multi-branch neural network, and/or other parameters. In some cases, compression is performed only on certain selected layers or branches of the machine learning model. Candidate filters from the selected layers or branches can be removed from the machine learning model in a way that preserves local features of the machine learning model.

    GENERATING CONTEXTUAL TAGS FOR DIGITAL CONTENT

    公开(公告)号:US20210034657A1

    公开(公告)日:2021-02-04

    申请号:US16525366

    申请日:2019-07-29

    Applicant: Adobe Inc.

    Abstract: Systems, methods, and non-transitory computer-readable media are disclosed for determining multi-term contextual tags for digital content and propagating the multi-term contextual tags to additional digital content. For instance, the disclosed systems can utilize search query supervision to determine and associate multi-term contextual tags (e.g., tags that represent a specific concept based on the order of the terms in the tag) with digital content. Furthermore, the disclosed systems can propagate the multi-term contextual tags determined for the digital content to additional digital content based on similarities between the digital content and additional digital content (e.g., utilizing clustering techniques). Additionally, the disclosed systems can provide digital content as search results based on the associated multi-term contextual tags.

    AUTOMATICALLY DETECTING USER-REQUESTED OBJECTS IN IMAGES

    公开(公告)号:US20210027083A1

    公开(公告)日:2021-01-28

    申请号:US16518810

    申请日:2019-07-22

    Applicant: Adobe Inc.

    Abstract: The present disclosure relates to an object selection system that accurately detects and optionally automatically selects user-requested objects (e.g., query objects) in digital images. For example, the object selection system builds and utilizes an object selection pipeline to determine which object detection neural network to utilize to detect a query object based on analyzing the object class of a query object. In particular, the object selection system can identify both known object classes as well as objects corresponding to unknown object classes.

Patent Agency Ranking