UTILIZING SOFT CLASSIFICATIONS TO SELECT INPUT PARAMETERS FOR SEGMENTATION ALGORITHMS AND IDENTIFY SEGMENTS OF THREE-DIMENSIONAL DIGITAL MODELS

    公开(公告)号:US20220207749A1

    公开(公告)日:2022-06-30

    申请号:US17655226

    申请日:2022-03-17

    Applicant: Adobe Inc.

    Abstract: The present disclosure includes methods and systems for identifying and manipulating a segment of a three-dimensional digital model based on soft classification of the three-dimensional digital model. In particular, one or more embodiments of the disclosed systems and methods identify a soft classification of a digital model and utilize the soft classification to tune segmentation algorithms. For example, the disclosed systems and methods can utilize a soft classification to select a segmentation algorithm from a plurality of segmentation algorithms, to combine segmentation parameters from a plurality of segmentation algorithms, and/or to identify input parameters for a segmentation algorithm. The disclosed systems and methods can utilize the tuned segmentation algorithms to accurately and efficiently identify a segment of a three-dimensional digital model.

    3D object reconstruction using photometric mesh representation

    公开(公告)号:US10769848B1

    公开(公告)日:2020-09-08

    申请号:US16421729

    申请日:2019-05-24

    Applicant: Adobe, Inc.

    Abstract: Techniques are disclosed for 3D object reconstruction using photometric mesh representations. A decoder is pretrained to transform points sampled from 2D patches of representative objects into 3D polygonal meshes. An image frame of the object is fed into an encoder to get an initial latent code vector. For each frame and camera pair from the sequence, a polygonal mesh is rendered at the given viewpoints. The mesh is optimized by creating a virtual viewpoint, rasterized to obtain a depth map. The 3D mesh projections are aligned by projecting the coordinates corresponding to the polygonal face vertices of the rasterized mesh to both selected viewpoints. The photometric error is determined from RGB pixel intensities sampled from both frames. Gradients from the photometric error are backpropagated into the vertices of the assigned polygonal indices by relating the barycentric coordinates of each image to update the latent code vector.

    REALISTICALLY ILLUMINATED VIRTUAL OBJECTS EMBEDDED WITHIN IMMERSIVE ENVIRONMENTS

    公开(公告)号:US20190228567A1

    公开(公告)日:2019-07-25

    申请号:US15877142

    申请日:2018-01-22

    Applicant: ADOBE INC.

    Abstract: Matching an illumination of an embedded virtual object (VO) with current environment illumination conditions provides an enhanced immersive experience to a user. To match the VO and environment illuminations, illumination basis functions are determined based on preprocessing image data, captured as a first combination of intensities of direct illumination sources illuminates the environment. Each basis function corresponds to one of the direct illumination sources. During the capture of runtime image data, a second combination of intensities illuminates the environment. An illumination-weighting vector is determined based on the runtime image data. The determination of the weighting vector accounts for indirect illumination sources, such as surface reflections. The weighting vector encodes a superposition of the basis functions that corresponds to the second combination of intensities. The method illuminates the VO based on the weighting vector. The resulting illumination of the VO matches the second combination of the intensities and surface reflections.

    Neural network based 3D object surface mapping

    公开(公告)号:US11869132B2

    公开(公告)日:2024-01-09

    申请号:US17537343

    申请日:2021-11-29

    CPC classification number: G06T15/04 G06N3/045 G06N3/08 G06T17/20

    Abstract: Certain aspects and features of this disclosure relate to neural network based 3D object surface mapping. In one example, a first representation of a first surface of a first 3D object and a second representation of a second surface of a second 3D object are produced. A surface mapping function is generated for mapping the first surface to the second surface. The surface mapping function is defined the representations and by a neural network model configured to map a first 2D representation of the first surface to a second 2D representation of the second surface. One or more features of the a first 3D mesh on the first surface can be applied to a second 3D mesh on the second surface using the surface mapping function to produce a modified second surface, which can be rendered through a user interface.

    SUBDIVIDING A THREE-DIMENSIONAL MESH UTILIZING A NEURAL NETWORK

    公开(公告)号:US20230267686A1

    公开(公告)日:2023-08-24

    申请号:US17821704

    申请日:2022-08-23

    Applicant: Adobe Inc.

    CPC classification number: G06T17/20 G06N3/08 G06T7/13 G06N3/02 G06T2207/20081

    Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing one or more neural networks to recursively subdivide a three-dimensional mesh according to local geometries of vertices in the three-dimensional mesh. For example, the disclosed system can determine a local geometry (e.g., a one-ring neighborhood of half-flaps) for each vertex in a three-dimensional mesh. For each subdivision iteration, the disclosed system can then utilize a neural network to determine displacement coordinates for existing vertices in the three-dimensional mesh and coordinates for new vertices added to edges between the existing vertices in the three-dimensional mesh in accordance with the local geometries of the existing vertices. Furthermore, the disclosed system can generate a subdivided three-dimensional mesh based on the determined displacement coordinates for the existing vertices and the determined coordinates for the new vertices.

    Subdividing a three-dimensional mesh utilizing a neural network

    公开(公告)号:US11423617B2

    公开(公告)日:2022-08-23

    申请号:US16863189

    申请日:2020-04-30

    Applicant: Adobe Inc.

    Abstract: Methods, systems, and non-transitory computer readable storage media are disclosed for utilizing one or more neural networks to recursively subdivide a three-dimensional mesh according to local geometries of vertices in the three-dimensional mesh. For example, the disclosed system can determine a local geometry (e.g., a one-ring neighborhood of half-flaps) for each vertex in a three-dimensional mesh. For each subdivision iteration, the disclosed system can then utilize a neural network to determine displacement coordinates for existing vertices in the three-dimensional mesh and coordinates for new vertices added to edges between the existing vertices in the three-dimensional mesh in accordance with the local geometries of the existing vertices. Furthermore, the disclosed system can generate a subdivided three-dimensional mesh based on the determined displacement coordinates for the existing vertices and the determined coordinates for the new vertices.

    Mixing segmentation algorithms utilizing soft classifications to identify segments of three-dimensional digital models

    公开(公告)号:US11315255B2

    公开(公告)日:2022-04-26

    申请号:US16907663

    申请日:2020-06-22

    Applicant: ADOBE INC.

    Abstract: The present disclosure includes methods and systems for identifying and manipulating a segment of a three-dimensional digital model based on soft classification of the three-dimensional digital model. In particular, one or more embodiments of the disclosed systems and methods identify a soft classification of a digital model and utilize the soft classification to tune segmentation algorithms. For example, the disclosed systems and methods can utilize a soft classification to select a segmentation algorithm from a plurality of segmentation algorithms, to combine segmentation parameters from a plurality of segmentation algorithms, and/or to identify input parameters for a segmentation algorithm. The disclosed systems and methods can utilize the tuned segmentation algorithms to accurately and efficiently identify a segment of a three-dimensional digital model.

Patent Agency Ranking