Enhanced video shot matching using generative adversarial networks

    公开(公告)号:US11158090B2

    公开(公告)日:2021-10-26

    申请号:US16692503

    申请日:2019-11-22

    Applicant: Adobe Inc.

    Abstract: This disclosure involves training generative adversarial networks to shot-match two unmatched images in a context-sensitive manner. For example, aspects of the present disclosure include accessing a trained generative adversarial network including a trained generator model and a trained discriminator model. A source image and a reference image may be inputted into the generator model to generate a modified source image. The modified source image and the reference image may be inputted into the discriminator model to determine a likelihood that the modified source image is color-matched with the reference image. The modified source image may be outputted as a shot-match with the reference image in response to determining, using the discriminator model, that the modified source image and the reference image are color-matched.

    Automatic Digital Parameter Adjustment Including Tone and Color Correction

    公开(公告)号:US20210160466A1

    公开(公告)日:2021-05-27

    申请号:US16696160

    申请日:2019-11-26

    Applicant: Adobe Inc.

    Abstract: Systems and techniques for automatic digital parameter adjustment are described that leverage insights learned from an image set to automatically predict parameter values for an input item of digital visual content. To do so, the automatic digital parameter adjustment techniques described herein captures visual and contextual features of digital visual content to determine balanced visual output in a range of visual scenes and settings. The visual and contextual features of digital visual content are used to train a parameter adjustment model through machine learning techniques that captures feature patterns and interactions. The parameter adjustment model exploits these feature interactions to determine visually pleasing parameter values for an input item of digital visual content. The predicted parameter values are output, allowing further adjustment to the parameter values.

    INTELLIGENT VIDEO REFRAMING
    43.
    发明申请

    公开(公告)号:US20200304754A1

    公开(公告)日:2020-09-24

    申请号:US16359876

    申请日:2019-03-20

    Applicant: ADOBE INC.

    Abstract: Embodiments of the present invention are directed towards reframing videos from one aspect ratio to another aspect ratio while maintaining visibility of regions of interest. A set of regions of interest are determined in frames in a video with a first aspect ratio. The set of regions of interest can be used to estimate an initial camera path. An optimal camera path is determined by leveraging the identified regions of interest using the initial camera path. Sub crops with a second aspect ratio different from the first aspect ratio of the video are identified. The sub crops are placed as designated using the optimal camera path to generate a cropped video with the second aspect ratio.

    IMAGE COMPOSITES USING A GENERATIVE NEURAL NETWORK

    公开(公告)号:US20200302251A1

    公开(公告)日:2020-09-24

    申请号:US16897068

    申请日:2020-06-09

    Applicant: Adobe Inc.

    Abstract: The present disclosure relates to an image composite system that employs a generative adversarial network to generate realistic composite images. For example, in one or more embodiments, the image composite system trains a geometric prediction neural network using an adversarial discrimination neural network to learn warp parameters that provide correct geometric alignment of foreground objects with respect to a background image. Once trained, the determined warp parameters provide realistic geometric corrections to foreground objects such that the warped foreground objects appear to blend into background images naturally when composited together.

    TRANSCRIPT-BASED INSERTION OF SECONDARY VIDEO CONTENT INTO PRIMARY VIDEO CONTENT

    公开(公告)号:US20200273493A1

    公开(公告)日:2020-08-27

    申请号:US16281903

    申请日:2019-02-21

    Applicant: Adobe Inc.

    Abstract: Certain embodiments involve transcript-based techniques for facilitating insertion of secondary video content into primary video content. For instance, a video editor presents a video editing interface having a primary video section displaying a primary video, a text-based navigation section having navigable portions of a primary video transcript, and a secondary video menu section displaying candidate secondary videos. In some embodiments, candidate secondary videos are obtained by using target terms detected in the transcript to query a remote data source for the candidate secondary videos. In embodiments involving video insertion, the video editor identifies a portion of the primary video corresponding to a portion of the transcript selected within the text-based navigation section. The video editor inserts a secondary video, which is selected from the candidate secondary videos based on an input received at the secondary video menu section, at the identified portion of the primary video.

    Generating spatial audio using a predictive model

    公开(公告)号:US10701303B2

    公开(公告)日:2020-06-30

    申请号:US15937349

    申请日:2018-03-27

    Applicant: Adobe Inc.

    Abstract: Certain embodiments involve generating and providing spatial audio using a predictive model. For example, a generates, using a predictive model, a visual representation of visual content provideable to a user device by encoding the visual content into the visual representation that indicates a visual element in the visual content. The system generates, using the predictive model, an audio representation of audio associated with the visual content by encoding the audio into the audio representation that indicates an audio element in the audio. The system also generates, using the predictive model, spatial audio based at least in part on the audio element and associating the spatial audio with the visual element. The system can also augment the visual content using the spatial audio by at least associating the spatial audio with the visual content.

    DEEP PATCH FEATURE PREDICTION FOR IMAGE INPAINTING

    公开(公告)号:US20190295227A1

    公开(公告)日:2019-09-26

    申请号:US15935994

    申请日:2018-03-26

    Applicant: Adobe Inc.

    Abstract: Techniques for using deep learning to facilitate patch-based image inpainting are described. In an example, a computer system hosts a neural network trained to generate, from an image, code vectors including features learned by the neural network and descriptive of patches. The image is received and contains a region of interest (e.g., a hole missing content). The computer system inputs it to the network and, in response, receives the code vectors. Each code vector is associated with a pixel in the image. Rather than comparing RGB values between patches, the computer system compares the code vector of a pixel inside the region to code vectors of pixels outside the region to find the best match based on a feature similarity measure (e.g., a cosine similarity). The pixel value of the pixel inside the region is set based on the pixel value of the matched pixel outside this region.

    IMAGE COMPOSITES USING A GENERATIVE ADVERSARIAL NEURAL NETWORK

    公开(公告)号:US20190251401A1

    公开(公告)日:2019-08-15

    申请号:US15897910

    申请日:2018-02-15

    Applicant: Adobe Inc.

    Abstract: The present disclosure relates to an image composite system that employs a generative adversarial network to generate realistic composite images. For example, in one or more embodiments, the image composite system trains a geometric prediction neural network using an adversarial discrimination neural network to learn warp parameters that provide correct geometric alignment of foreground objects with respect to a background image. Once trained, the determined warp parameters provide realistic geometric corrections to foreground objects such that the warped foreground objects appear to blend into background images naturally when composited together.

    Visual odometry using object priors

    公开(公告)号:US10204423B2

    公开(公告)日:2019-02-12

    申请号:US15430659

    申请日:2017-02-13

    Applicant: Adobe Inc.

    Abstract: Disclosed are techniques for more accurately estimating the pose of a camera used to capture a three-dimensional scene. Accuracy is enhanced by leveraging three-dimensional object priors extracted from a large-scale three-dimensional shape database. This allows existing feature matching techniques to be augmented by generic three-dimensional object priors, thereby providing robust information about object orientations across multiple images or frames. More specifically, the three-dimensional object priors provide a unit that is easier and more reliably tracked between images than a single feature point. By adding object pose estimates across images, drift is reduced and the resulting visual odometry techniques are more robust and accurate. This eliminates the need for three-dimensional object templates that are specifically generated for the imaged object, training data obtained for a specific environment, and other tedious preprocessing steps. Entire object classes identified in a three-dimensional shape database can be used to train an object detector.

Patent Agency Ranking