-
公开(公告)号:US20210279952A1
公开(公告)日:2021-09-09
申请号:US17193405
申请日:2021-03-05
Applicant: Nvidia Corporation
Inventor: Wenzheng Chen , Yuxuan Zhang , Sanja Fidler , Huan Ling , Jun Gao , Antonio Torralba Barriuso
Abstract: Approaches are presented for training an inverse graphics network. An image synthesis network can generate training data for an inverse graphics network. In turn, the inverse graphics network can teach the synthesis network about the physical three-dimensional (3D) controls. Such an approach can provide for accurate 3D reconstruction of objects from 2D images using the trained inverse graphics network, while requiring little annotation of the provided training data. Such an approach can extract and disentangle 3D knowledge learned by generative models by utilizing differentiable renderers, enabling a disentangled generative model to function as a controllable 3D “neural renderer,” complementing traditional graphics renderers.
-
12.
公开(公告)号:US20240296627A1
公开(公告)日:2024-09-05
申请号:US18662020
申请日:2024-05-13
Applicant: NVIDIA Corporation
Inventor: Tianchang Shen , Jun Gao , Kangxue Yin , Ming-Yu Liu , Sanja Fidler
CPC classification number: G06T17/20 , G06T7/50 , G06T2207/10028 , G06T2207/20081 , G06T2207/20084
Abstract: In various examples, a deep three-dimensional (3D) conditional generative model is implemented that can synthesize high resolution 3D shapes using simple guides—such as coarse voxels, point clouds, etc.—by marrying implicit and explicit 3D representations into a hybrid 3D representation. The present approach may directly optimize for the reconstructed surface, allowing for the synthesis of finer geometric details with fewer artifacts. The systems and methods described herein may use a deformable tetrahedral grid that encodes a discretized signed distance function (SDF) and a differentiable marching tetrahedral layer that converts the implicit SDF representation to an explicit surface mesh representation. This combination allows joint optimization of the surface geometry and topology as well as generation of the hierarchy of subdivisions using reconstruction and adversarial losses defined explicitly on the surface mesh.
-
公开(公告)号:US20240161403A1
公开(公告)日:2024-05-16
申请号:US18232279
申请日:2023-08-09
Applicant: NVIDIA Corporation
Inventor: Chen-Hsuan Lin , Tsung-Yi Lin , Ming-Yu Liu , Sanja Fidler , Karsten Kreis , Luming Tang , Xiaohui Zeng , Jun Gao , Xun Huang , Towaki Takikawa
CPC classification number: G06T17/20 , G06T3/40 , G06T15/04 , G06T17/005 , G06T19/20
Abstract: Text-to-image generation generally refers to the process of generating an image from one or more text prompts input by a user. While artificial intelligence has been a valuable tool for text-to-image generation, current artificial intelligence-based solutions are more limited as it relates to text-to-3D content creation. For example, these solutions are oftentimes category-dependent, or synthesize 3D content at a low resolution. The present disclosure provides a process and architecture for high-resolution text-to-3D content creation.
-
公开(公告)号:US11967024B2
公开(公告)日:2024-04-23
申请号:US17827918
申请日:2022-05-30
Applicant: NVIDIA Corporation
Inventor: Carl Jacob Munkberg , Jon Niklas Theodor Hasselgren , Tianchang Shen , Jun Gao , Wenzheng Chen , Alex John Bauld Evans , Thomas Müller-Höhne , Sanja Fidler
CPC classification number: G06T17/205 , G06N3/084 , G06T9/002 , G06T15/04 , G06T15/506 , G06T19/00 , G06T2210/36
Abstract: A technique is described for extracting or constructing a three-dimensional (3D) model from multiple two-dimensional (2D) images. In an embodiment, a foreground segmentation mask or depth field may be provided as an additional supervision input with each 2D image. In an embodiment, the foreground segmentation mask or depth field is automatically generated for each 2D image. The constructed 3D model comprises a triangular mesh topology, materials, and environment lighting. The constructed 3D model is represented in a format that can be directly edited and/or rendered by conventional application programs, such as digital content creation (DCC) tools. For example, the constructed 3D model may be represented as a triangular surface mesh (with arbitrary topology), a set of 2D textures representing spatially-varying material parameters, and an environment map. Furthermore, the constructed 3D model may be included in 3D scenes and interacts realistically with other objects.
-
公开(公告)号:US20230134690A1
公开(公告)日:2023-05-04
申请号:US17981770
申请日:2022-11-07
Applicant: Nvidia Corporation
Inventor: Wenzheng Chen , Yuxuan Zhang , Sanja Fidler , Huan Ling , Jun Gao , Antonio Torralba Barriuso
Abstract: Approaches are presented for training an inverse graphics network. An image synthesis network can generate training data for an inverse graphics network. In turn, the inverse graphics network can teach the synthesis network about the physical three-dimensional (3D) controls. Such an approach can provide for accurate 3D reconstruction of objects from 2D images using the trained inverse graphics network, while requiring little annotation of the provided training data. Such an approach can extract and disentangle 3D knowledge learned by generative models by utilizing differentiable renderers, enabling a disentangled generative model to function as a controllable 3D “neural renderer,” complementing traditional graphics renderers.
-
公开(公告)号:US20250124640A1
公开(公告)日:2025-04-17
申请号:US18486046
申请日:2023-10-12
Applicant: NVIDIA Corporation
Inventor: Jonathan Peter Lorraine , Cheng (Kevin) Xie , Xiaohui Zeng , Jun Gao , Sanja Fidler , James Lucas
Abstract: Apparatuses, systems, and techniques to train one or more neural networks using stratified sampled training data parameters. In at least one embodiment, one or more stochastic training data parameters may be stratified sampled from one or more sampling ranges to compute a gradient for updating the one or more neural networks.
-
公开(公告)号:US20250095275A1
公开(公告)日:2025-03-20
申请号:US18630480
申请日:2024-04-09
Applicant: NVIDIA Corporation
Inventor: Zian Wang , Tianchang Shen , Jun Gao , Merlin Nimier-David , Thomas Müller-Höhne , Alexander Keller , Sanja Fidler , Zan Gojcic , Nicholas Mark Worth Sharp
Abstract: In various examples, images (e.g., novel views) of an object may be rendered using an optimized number of samples of a 3D representation of the object. The optimized number of the samples may be determined based at least on casting rays into a scene that includes the 3D representation of the object and/or an acceleration data structure corresponding to the object. The acceleration data structure may include features corresponding to characteristics of the object, and the features may be indicative of the number of samples to be obtained from various portions of the 3D representation of the object to render the images. In some examples, the 3D representation may be a neural radiance field that includes, as a neural output, a spatially varying kernel size predicting the characteristics of the object, and the features of the acceleration data structure may be related to the spatially varying kernel size.
-
公开(公告)号:US12243152B2
公开(公告)日:2025-03-04
申请号:US18441486
申请日:2024-02-14
Applicant: NVIDIA Corporation
Inventor: Wenzheng Chen , Joey Litalien , Jun Gao , Zian Wang , Clement Tse Tsian Christophe Louis Fuji Tsang , Sameh Khamis , Or Litany , Sanja Fidler
Abstract: In various examples, information may be received for a 3D model, such as 3D geometry information, lighting information, and material information. A machine learning model may be trained to disentangle the 3D geometry information, the lighting information, and/or material information from input data to provide the information, which may be used to project geometry of the 3D model onto an image plane to generate a mapping between pixels and portions of the 3D model. Rasterization may then use the mapping to determine which pixels are covered and in what manner, by the geometry. The mapping may also be used to compute radiance for points corresponding to the one or more 3D models using light transport simulation. Disclosed approaches may be used in various applications, such as image editing, 3D model editing, synthetic data generation, and/or data set augmentation.
-
公开(公告)号:US12112445B2
公开(公告)日:2024-10-08
申请号:US17467792
申请日:2021-09-07
Applicant: Nvidia Corporation
Inventor: Kangxue Yin , Jun Gao , Masha Shugrina , Sameh Khamis , Sanja Fidler
CPC classification number: G06T19/20 , G06N3/045 , G06T3/02 , G06T3/18 , G06T7/11 , G06T15/04 , G06T17/20 , G06T2200/04 , G06T2207/20084 , G06T2219/2021
Abstract: Generation of three-dimensional (3D) object models may be challenging for users without a sufficient skill set for content creation and may also be resource intensive. One or more style transfer networks may be used for part-aware style transformation of both geometric features and textural components of a source asset to a target asset. The source asset may be segmented into particular parts and then ellipsoid approximations may be warped according to correspondence of the particular parts to the target assets. Moreover, a texture associated with the target asset may be used to warp or adjust a source texture, where the new texture can be applied to the warped parts.
-
公开(公告)号:US20220392162A1
公开(公告)日:2022-12-08
申请号:US17718172
申请日:2022-04-11
Applicant: NVIDIA Corporation
Inventor: Tianchang Shen , Jun Gao , Kangxue Yin , Ming-Yu Liu , Sanja Fidler
Abstract: In various examples, a deep three-dimensional (3D) conditional generative model is implemented that can synthesize high resolution 3D shapes using simple guides—such as coarse voxels, point clouds, etc.—by marrying implicit and explicit 3D representations into a hybrid 3D representation. The present approach may directly optimize for the reconstructed surface, allowing for the synthesis of finer geometric details with fewer artifacts. The systems and methods described herein may use a deformable tetrahedral grid that encodes a discretized signed distance function (SDF) and a differentiable marching tetrahedral layer that converts the implicit SDF representation to an explicit surface mesh representation. This combination allows joint optimization of the surface geometry and topology as well as generation of the hierarchy of subdivisions using reconstruction and adversarial losses defined explicitly on the surface mesh.
-
-
-
-
-
-
-
-
-