-
公开(公告)号：US11704865B2
公开(公告)日：2023-07-18
申请号：US17383294
申请日：2021-07-22
Applicant: Adobe Inc.
Inventor： Ruben Villegas , Yunseok Jang , Duygu Ceylan Aksit , Jimei Yang , Xin Sun
Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate realistic shading for three-dimensional objects inserted into digital images. The disclosed system utilizes a light encoder neural network to generate a representation embedding of lighting in a digital image. Additionally, the disclosed system determines points of the three-dimensional object visible within a camera view. The disclosed system generates a self-occlusion map for the digital three-dimensional object by determining whether fixed sets of rays uniformly sampled from the points intersects with the digital three-dimensional object. The disclosed system utilizes a generator neural network to determine a shading map for the digital three-dimensional object based on the representation embedding of lighting in the digital image and the self-occlusion map. Additionally, the disclosed system generates a modified digital image with the three-dimensional object inserted into the digital image with consistent lighting of the three-dimensional object and the digital image.
-
公开(公告)号：US20230132642A1
公开(公告)日：2023-05-04
申请号：US17515079
申请日：2021-10-29
Applicant: Adobe Inc.
Inventor： Qi Sun , Xin Sun , Stefano Petrangeli , Shaoyu Chen , Li-Yi Wei , Jose Ignacio Echevarria Vallespi
Abstract: A method includes accessing a first object in a virtual environment, the first object representing a first asset at a first level of detail (LoD). A second object is generated to represent the first asset at a second LoD having decreased complexity. The method further includes determining a first importance value for the first asset and, based on the first importance value, selecting the first object to represent the first asset. Additionally, the method includes accessing a third object representing the second asset at the first LoD and generating a fourth object representing the second asset at the second LoD. The method further includes determining a second importance value, lower than the first importance value, for the second asset and selecting the fourth object to represent the second asset. The method further includes causing a client device to update a display of the virtual environment by transmitting the selected objects.
-
公开(公告)号：US20220130087A1
公开(公告)日：2022-04-28
申请号：US17082378
申请日：2020-10-28
Applicant: Adobe Inc.
Inventor： Xin Sun , Vineet Batra , Sumit Dhingra , Nathan Aaron Carr , Ankit Phogat
Abstract: A multi-layer light source includes an emissive layer and a textured lighting gel layer, the lighting gel layer being situated between the emissive layer and a 2D canvas or a 3D object. User inputs controlling the multi-layer light source are received, these user inputs being provided with the user interacting with the 2D canvas without switching to editing in 3D space. The multi-layer light source is configured based on the user inputs and, based on the configuration, emission of light rays from the multi-layer light source is determined. Areas of shadows cast by 3D objects are also determined. An image generation system determines, a color of a location (e.g., a pixel) on the 2D canvas or the 3D object that a light ray intersects based on the color that is in the lighting gel layer that the light ray passes through.
-
公开(公告)号：US20210319256A1
公开(公告)日：2021-10-14
申请号：US17332773
申请日：2021-05-27
Applicant: Adobe Inc.
Inventor： Xin Sun , Sohrab Amirghodsi , Nathan Carr , Michal Lukac
Abstract: The present disclosure is directed toward systems, methods, and non-transitory computer readable media for generating a modified digital image by identifying patch matches within a digital image utilizing a Gaussian mixture model. For example, the systems described herein can identify sample patches and corresponding matching portions within a digital image. The systems can also identify transformations between the sample patches and the corresponding matching portions. Based on the transformations, the systems can generate a Gaussian mixture model, and the systems can modify a digital image by replacing a target region with target matching portions identified in accordance with the Gaussian mixture model.
-
公开(公告)号：US20200082610A1
公开(公告)日：2020-03-12
申请号：US16126552
申请日：2018-09-10
Applicant: Adobe Inc.
Inventor： Xin Sun , Zhili Chen , Nathan Carr , Julio Marco Murria , Jimei Yang
Abstract: According to one general aspect, systems and techniques for rendering a painting stroke of a three-dimensional digital painting include receiving a painting stroke input on a canvas, where the painting stroke includes a plurality of pixels. For each of the pixels in the plurality of pixels, a neighborhood patch of pixels is selected and input into a neural network and a shading function is output from the neural network. The painting stroke is rendered on the canvas using the shading function.
-
Patent Agency Ranking
公开(公告)号：US20190228567A1
公开(公告)日：2019-07-25
申请号：US15877142
申请日：2018-01-22
Applicant: ADOBE INC.
Inventor： Jeong Joon Park , Zhili Chen , Xin Sun , Vladimir Kim , Kalyan Krishna Sunkavalli , Duygu Ceylan Aksit
Abstract: Matching an illumination of an embedded virtual object (VO) with current environment illumination conditions provides an enhanced immersive experience to a user. To match the VO and environment illuminations, illumination basis functions are determined based on preprocessing image data, captured as a first combination of intensities of direct illumination sources illuminates the environment. Each basis function corresponds to one of the direct illumination sources. During the capture of runtime image data, a second combination of intensities illuminates the environment. An illumination-weighting vector is determined based on the runtime image data. The determination of the weighting vector accounts for indirect illumination sources, such as surface reflections. The weighting vector encodes a superposition of the basis functions that corresponds to the second combination of intensities. The method illuminates the VO based on the weighting vector. The resulting illumination of the VO matches the second combination of the intensities and surface reflections.
-
公开(公告)号：US12148089B2
公开(公告)日：2024-11-19
申请号：US17889168
申请日：2022-08-16
Applicant: Adobe Inc.
Inventor： Ankit Phogat , Xin Sun , Vineet Batra , Sumit Dhingra , Nathan A. Carr , Milos Hasan
Abstract: Embodiments are disclosed for performing 3-D vectorization. The method includes obtaining a three-dimensional rendered image and a camera position. The method further includes obtaining a triangle mesh representing the three-dimensional rendered image. The method further involves creating a reduced triangle mesh by removing one or more triangles from the triangle mesh. The method further involves subdividing each triangle of the reduced triangle mesh into one or more subdivided triangles. The method further involves performing a mapping of each pixel of the three-dimensional rendered image to the reduced triangle mesh. The method further involves assigning a color value to each vertex of the reduced triangle mesh. The method further involves sorting each triangle of the reduced triangle mesh using a depth value of each triangle. The method further involves generating a two-dimensional triangle mesh using the sorted triangles of the reduced triangle mesh.
-
公开(公告)号：US11657562B2
公开(公告)日：2023-05-23
申请号：US17233910
申请日：2021-04-19
Applicant: Adobe Inc.
Inventor： Xin Sun , Milos Hasan , Nathan Carr
CPC classification number: G06T15/06 , G06T1/20 , G06T7/507 , G06T7/536 , G06T15/005
Abstract: The present disclosure relates to systems, non-transitory computer-readable media, and methods that utilize hemispherical clamping for importance sampling of an image-based light (IBL) to generate a digital image of a virtual environment. For example, the disclosed systems identify a hemispherical portion of an IBL image that corresponds to a reflective surface location on a virtual object. The disclosed systems can then clamp the IBL image using one or more importance sampling algorithms to exclude portions of the IBL image outside of the hemispherical portion that do not contribute direct lighting onto the reflective surface location. The disclosed systems can further utilize the one or more importance sampling algorithms to efficiently sample a ray direction between the reflective surface location and the hemispherical portion of the IBL image. In certain embodiments, the disclosed systems use the sampled ray direction to generate a digital image rendering portraying the virtual object.
-
公开(公告)号：US20220284640A1
公开(公告)日：2022-09-08
申请号：US17664800
申请日：2022-05-24
Applicant: Adobe Inc.
Inventor： Xin Sun , Ruben Villegas , Manuel Lagunas Arto , Jimei Yang , Jianming Zhang
Abstract: Introduced here are techniques for relighting an image by automatically segmenting a human object in an image. The segmented image is input to an encoder that transforms it into a feature space. The feature space is concatenated with coefficients of a target illumination for the image and input to an albedo decoder and a light transport detector to predict an albedo map and a light transport matrix, respectively. In addition, the output of the encoder is concatenated with outputs of residual parts of each decoder and fed to a light coefficients block, which predicts coefficients of the illumination for the image. The light transport matrix and predicted illumination coefficients are multiplied to obtain a shading map that can sharpen details of the image. Scaling the resulting image by the albedo map to produce the relight image. The relight image can be refined to denoise the relight image.
-
公开(公告)号：US20210295571A1
公开(公告)日：2021-09-23
申请号：US16823092
申请日：2020-03-18
Applicant: Adobe Inc.
Inventor： Xin Sun , Ruben Villegas , Manuel Lagunas Arto , Jimei Yang , Jianming Zhang
Abstract: Introduced here are techniques for relighting an image by automatically segmenting a human object in an image. The segmented image is input to an encoder that transforms it into a feature space. The feature space is concatenated with coefficients of a target illumination for the image and input to an albedo decoder and a light transport detector to predict an albedo map and a light transport matrix, respectively. In addition, the output of the encoder is concatenated with outputs of residual parts of each decoder and fed to a light coefficients block, which predicts coefficients of the illumination for the image. The light transport matrix and predicted illumination coefficients are multiplied to obtain a shading map that can sharpen details of the image. Scaling the resulting image by the albedo map to produce the relight image. The relight image can be refined to denoise the relight image.
-
-
-
-
-
-
-
-
-
Search Results
38
records
(took 0.022 seconds)
