Large-scale outdoor augmented reality scenes using camera pose based on learned descriptors

    公开(公告)号:US11568642B2

    公开(公告)日:2023-01-31

    申请号:US17068429

    申请日:2020-10-12

    申请人: ADOBE INC.

    IPC分类号: G06V20/20 G06N20/00 G06T7/70

    摘要: Methods and systems are provided for facilitating large-scale augmented reality in relation to outdoor scenes using estimated camera pose information. In particular, camera pose information for an image can be estimated by matching the image to a rendered ground-truth terrain model with known camera pose information. To match images with such renders, data driven cross-domain feature embedding can be learned using a neural network. Cross-domain feature descriptors can be used for efficient and accurate feature matching between the image and the terrain model renders. This feature matching allows images to be localized in relation to the terrain model, which has known camera pose information. This known camera pose information can then be used to estimate camera pose information in relation to the image.

    RECONSTRUCTING THREE-DIMENSIONAL SCENES PORTRAYED IN DIGITAL IMAGES UTILIZING POINT CLOUD MACHINE-LEARNING MODELS

    公开(公告)号:US20220277514A1

    公开(公告)日:2022-09-01

    申请号:US17186522

    申请日:2021-02-26

    申请人: Adobe Inc.

    摘要: This disclosure describes implementations of a three-dimensional (3D) scene recovery system that reconstructs a 3D scene representation of a scene portrayed in a single digital image. For instance, the 3D scene recovery system trains and utilizes a 3D point cloud model to recover accurate intrinsic camera parameters from a depth map of the digital image. Additionally, the 3D point cloud model may include multiple neural networks that target specific intrinsic camera parameters. For example, the 3D point cloud model may include a depth 3D point cloud neural network that recovers the depth shift as well as include a focal length 3D point cloud neural network that recovers the camera focal length. Further, the 3D scene recovery system may utilize the recovered intrinsic camera parameters to transform the single digital image into an accurate and realistic 3D scene representation, such as a 3D point cloud.

    Automated Digital Parameter Adjustment for Digital Images

    公开(公告)号:US20220182588A1

    公开(公告)日:2022-06-09

    申请号:US17526998

    申请日:2021-11-15

    申请人: Adobe Inc.

    IPC分类号: H04N9/31 G06T5/00 H04N9/73

    摘要: Systems and techniques for automatic digital parameter adjustment are described that leverage insights learned from an image set to automatically predict parameter values for an input item of digital visual content. To do so, the automatic digital parameter adjustment techniques described herein captures visual and contextual features of digital visual content to determine balanced visual output in a range of visual scenes and settings. The visual and contextual features of digital visual content are used to train a parameter adjustment model through machine learning techniques that captures feature patterns and interactions. The parameter adjustment model exploits these feature interactions to determine visually pleasing parameter values for an input item of digital visual content. The predicted parameter values are output, allowing further adjustment to the parameter values.

    Temporally distributed neural networks for video semantic segmentation

    公开(公告)号:US11354906B2

    公开(公告)日:2022-06-07

    申请号:US16846544

    申请日:2020-04-13

    申请人: Adobe Inc.

    摘要: A Video Semantic Segmentation System (VSSS) is disclosed that performs accurate and fast semantic segmentation of videos using a set of temporally distributed neural networks. The VSSS receives as input a video signal comprising a contiguous sequence of temporally-related video frames. The VSSS extracts features from the video frames in the contiguous sequence and based upon the extracted features, selects, from a set of labels, a label to be associated with each pixel of each video frame in the video signal. In certain embodiments, a set of multiple neural networks are used to extract the features to be used for video segmentation and the extraction of features is distributed among the multiple neural networks in the set. A strong feature representation representing the entirety of the features is produced for each video frame in the sequence of video frames by aggregating the output features extracted by the multiple neural networks.

    Dynamically rendering 360-degree videos using view-specific-filter parameters

    公开(公告)号:US11178374B2

    公开(公告)日:2021-11-16

    申请号:US16428201

    申请日:2019-05-31

    申请人: Adobe Inc.

    摘要: This disclosure relates to methods, non-transitory computer readable media, and systems that generate and dynamically change filter parameters for a frame of a 360-degree video based on detecting a field of view from a computing device. As a computing device rotates or otherwise changes orientation, for instance, the disclosed systems can detect a field of view and interpolate one or more filter parameters corresponding to nearby spatial keyframes of the 360-degree video to generate view-specific-filter parameters. By generating and storing filter parameters for spatial keyframes corresponding to different times and different view directions, the disclosed systems can dynamically adjust color grading or other visual effects using interpolated, view-specific-filter parameters to render a filtered version of the 360-degree video.

    Enhanced video shot matching using generative adversarial networks

    公开(公告)号:US11158090B2

    公开(公告)日:2021-10-26

    申请号:US16692503

    申请日:2019-11-22

    申请人: Adobe Inc.

    摘要: This disclosure involves training generative adversarial networks to shot-match two unmatched images in a context-sensitive manner. For example, aspects of the present disclosure include accessing a trained generative adversarial network including a trained generator model and a trained discriminator model. A source image and a reference image may be inputted into the generator model to generate a modified source image. The modified source image and the reference image may be inputted into the discriminator model to determine a likelihood that the modified source image is color-matched with the reference image. The modified source image may be outputted as a shot-match with the reference image in response to determining, using the discriminator model, that the modified source image and the reference image are color-matched.

    Automatic Digital Parameter Adjustment Including Tone and Color Correction

    公开(公告)号:US20210160466A1

    公开(公告)日:2021-05-27

    申请号:US16696160

    申请日:2019-11-26

    申请人: Adobe Inc.

    IPC分类号: H04N9/31 H04N9/73 G06T5/00

    摘要: Systems and techniques for automatic digital parameter adjustment are described that leverage insights learned from an image set to automatically predict parameter values for an input item of digital visual content. To do so, the automatic digital parameter adjustment techniques described herein captures visual and contextual features of digital visual content to determine balanced visual output in a range of visual scenes and settings. The visual and contextual features of digital visual content are used to train a parameter adjustment model through machine learning techniques that captures feature patterns and interactions. The parameter adjustment model exploits these feature interactions to determine visually pleasing parameter values for an input item of digital visual content. The predicted parameter values are output, allowing further adjustment to the parameter values.

    INTELLIGENT VIDEO REFRAMING
    8.
    发明申请

    公开(公告)号:US20200304754A1

    公开(公告)日:2020-09-24

    申请号:US16359876

    申请日:2019-03-20

    申请人: ADOBE INC.

    IPC分类号: H04N7/01 H04N5/262

    摘要: Embodiments of the present invention are directed towards reframing videos from one aspect ratio to another aspect ratio while maintaining visibility of regions of interest. A set of regions of interest are determined in frames in a video with a first aspect ratio. The set of regions of interest can be used to estimate an initial camera path. An optimal camera path is determined by leveraging the identified regions of interest using the initial camera path. Sub crops with a second aspect ratio different from the first aspect ratio of the video are identified. The sub crops are placed as designated using the optimal camera path to generate a cropped video with the second aspect ratio.

    IMAGE COMPOSITES USING A GENERATIVE NEURAL NETWORK

    公开(公告)号:US20200302251A1

    公开(公告)日:2020-09-24

    申请号:US16897068

    申请日:2020-06-09

    申请人: Adobe Inc.

    IPC分类号: G06K9/66 G06N3/04 G06N3/08

    摘要: The present disclosure relates to an image composite system that employs a generative adversarial network to generate realistic composite images. For example, in one or more embodiments, the image composite system trains a geometric prediction neural network using an adversarial discrimination neural network to learn warp parameters that provide correct geometric alignment of foreground objects with respect to a background image. Once trained, the determined warp parameters provide realistic geometric corrections to foreground objects such that the warped foreground objects appear to blend into background images naturally when composited together.

    TRANSCRIPT-BASED INSERTION OF SECONDARY VIDEO CONTENT INTO PRIMARY VIDEO CONTENT

    公开(公告)号:US20200273493A1

    公开(公告)日:2020-08-27

    申请号:US16281903

    申请日:2019-02-21

    申请人: Adobe Inc.

    摘要: Certain embodiments involve transcript-based techniques for facilitating insertion of secondary video content into primary video content. For instance, a video editor presents a video editing interface having a primary video section displaying a primary video, a text-based navigation section having navigable portions of a primary video transcript, and a secondary video menu section displaying candidate secondary videos. In some embodiments, candidate secondary videos are obtained by using target terms detected in the transcript to query a remote data source for the candidate secondary videos. In embodiments involving video insertion, the video editor identifies a portion of the primary video corresponding to a portion of the transcript selected within the text-based navigation section. The video editor inserts a secondary video, which is selected from the candidate secondary videos based on an input received at the secondary video menu section, at the identified portion of the primary video.