-
公开(公告)号：US20240371096A1
公开(公告)日：2024-11-07
申请号：US18312102
申请日：2023-05-04
Applicant: Nvidia Corporation
Inventor： Sameh Khamis , Koki Nagano , Jan Kautz , Sanja Fidler
Abstract: Approaches presented herein provide systems and methods for disentangling identity from expression input models. One or more machine learning systems may be trained directly from three-dimensional (3D) points to develop unique latent codes for expressions associated with different identities. These codes may then be mapped to different identities to independently model an object, such as a face, to generate a new mesh including an expression for an independent identity. A pipeline may include a set of machine learning systems to determine model parameters and also adjust input expression codes using gradient backpropagation in order train models for incorporation into a content development pipeline.
-
2.发明公开公开(公告)号：US20240185506A1
公开(公告)日：2024-06-06
申请号：US18441486
申请日：2024-02-14
Applicant: NVIDIA Corporation
Inventor： Wenzheng Chen , Joey Litalien , Jun Gao , Zian Wang , Clement Tse Tsian Christophe Louis Fuji Tsang , Sameh Khamis , Or Litany , Sanja Fidler
CPC classification number: G06T15/06 , G06T15/506 , G06T19/20 , G06T2219/2012
Abstract: In various examples, information may be received for a 3D model, such as 3D geometry information, lighting information, and material information. A machine learning model may be trained to disentangle the 3D geometry information, the lighting information, and/or material information from input data to provide the information, which may be used to project geometry of the 3D model onto an image plane to generate a mapping between pixels and portions of the 3D model. Rasterization may then use the mapping to determine which pixels are covered and in what manner, by the geometry. The mapping may also be used to compute radiance for points corresponding to the one or more 3D models using light transport simulation. Disclosed approaches may be used in various applications, such as image editing, 3D model editing, synthetic data generation, and/or data set augmentation.
-
公开(公告)号：US20250029351A1
公开(公告)日：2025-01-23
申请号：US18905841
申请日：2024-10-03
Applicant: Nvidia Corporation
Inventor： Kangxue Yin , Jun Gao , Masha Shugrina , Sameh Khamis , Sanja Fidler
Abstract: Generation of three-dimensional (3D) object models may be challenging for users without a sufficient skill set for content creation and may also be resource intensive. One or more style transfer networks may be used for part-aware style transformation of both geometric features and textural components of a source asset to a target asset. The source asset may be segmented into particular parts and then ellipsoid approximations may be warped according to correspondence of the particular parts to the target assets. Moreover, a texture associated with the target asset may be used to warp or adjust a source texture, where the new texture can be applied to the warped parts.
-
公开(公告)号：US11922558B2
公开(公告)日：2024-03-05
申请号：US17826611
申请日：2022-05-27
Applicant: NVIDIA Corporation
Inventor： Wenzheng Chen , Joey Litalien , Jun Gao , Zian Wang , Clement Tse Tsian Christophe Louis Fuji Tsang , Sameh Khamis , Or Litany , Sanja Fidler
CPC classification number: G06T15/06 , G06T15/506 , G06T19/20 , G06T2219/2012
Abstract: In various examples, information may be received for a 3D model, such as 3D geometry information, lighting information, and material information. A machine learning model may be trained to disentangle the 3D geometry information, the lighting information, and/or material information from input data to provide the information, which may be used to project geometry of the 3D model onto an image plane to generate a mapping between pixels and portions of the 3D model. Rasterization may then use the mapping to determine which pixels are covered and in what manner, by the geometry. The mapping may also be used to compute radiance for points corresponding to the one or more 3D models using light transport simulation. Disclosed approaches may be used in various applications, such as image editing, 3D model editing, synthetic data generation, and/or data set augmentation.
-
公开(公告)号：US20230074420A1
公开(公告)日：2023-03-09
申请号：US17467792
申请日：2021-09-07
Applicant: Nvidia Corporation
Inventor： Kangxue Yin , Jun Gao , Masha Shugrina , Sameh Khamis , Sanja Fidler
Abstract: Generation of three-dimensional (3D) object models may be challenging for users without a sufficient skill set for content creation and may also be resource intensive. One or more style transfer networks may be used for part-aware style transformation of both geometric features and textural components of a source asset to a target asset. The source asset may be segmented into particular parts and then ellipsoid approximations may be warped according to correspondence of the particular parts to the target assets. Moreover, a texture associated with the target asset may be used to warp or adjust a source texture, where the new texture can be applied to the warped parts.
-
Patent Agency Ranking
公开(公告)号：US20240290054A1
公开(公告)日：2024-08-29
申请号：US18174863
申请日：2023-02-27
Applicant: Nvidia Corporation
Inventor： Kangxue Yin , Huan Ling , Masha Shugrina , Sameh Khamis , Sanja Fidler
IPC: G06T19/20 , G06N3/0475 , G06N3/08 , G06T15/04 , G06T15/10
CPC classification number: G06T19/20 , G06N3/0475 , G06N3/08 , G06T15/04 , G06T15/10 , G06T2219/2021 , G06T2219/2024
Abstract: Generation of three-dimensional (3D) object models may be challenging for users without a sufficient skill set for content creation and may also be resource intensive. One or more style transfer networks may be combined with a generative network to generate objects based on parameters associated with a textual input. An input including a 3D mesh and texture may be provided to a trained system along with a textual input that includes parameters for object generation. Features of the input object may be identified and then tuned in accordance with the textual input to generate a modified 3D object that includes a new texture along with one or more geometric adjustments.
-
公开(公告)号：US12243152B2
公开(公告)日：2025-03-04
申请号：US18441486
申请日：2024-02-14
Applicant: NVIDIA Corporation
Inventor： Wenzheng Chen , Joey Litalien , Jun Gao , Zian Wang , Clement Tse Tsian Christophe Louis Fuji Tsang , Sameh Khamis , Or Litany , Sanja Fidler
Abstract: In various examples, information may be received for a 3D model, such as 3D geometry information, lighting information, and material information. A machine learning model may be trained to disentangle the 3D geometry information, the lighting information, and/or material information from input data to provide the information, which may be used to project geometry of the 3D model onto an image plane to generate a mapping between pixels and portions of the 3D model. Rasterization may then use the mapping to determine which pixels are covered and in what manner, by the geometry. The mapping may also be used to compute radiance for points corresponding to the one or more 3D models using light transport simulation. Disclosed approaches may be used in various applications, such as image editing, 3D model editing, synthetic data generation, and/or data set augmentation.
-
公开(公告)号：US12112445B2
公开(公告)日：2024-10-08
申请号：US17467792
申请日：2021-09-07
Applicant: Nvidia Corporation
Inventor： Kangxue Yin , Jun Gao , Masha Shugrina , Sameh Khamis , Sanja Fidler
CPC classification number: G06T19/20 , G06N3/045 , G06T3/02 , G06T3/18 , G06T7/11 , G06T15/04 , G06T17/20 , G06T2200/04 , G06T2207/20084 , G06T2219/2021
Abstract: Generation of three-dimensional (3D) object models may be challenging for users without a sufficient skill set for content creation and may also be resource intensive. One or more style transfer networks may be used for part-aware style transformation of both geometric features and textural components of a source asset to a target asset. The source asset may be segmented into particular parts and then ellipsoid approximations may be warped according to correspondence of the particular parts to the target assets. Moreover, a texture associated with the target asset may be used to warp or adjust a source texture, where the new texture can be applied to the warped parts.
-
公开(公告)号：US20230081641A1
公开(公告)日：2023-03-16
申请号：US17551046
申请日：2021-12-14
Applicant: NVIDIA Corporation
Inventor： Koki Nagano , Eric Ryan Chan , Sameh Khamis , Shalini De Mello , Tero Tapani Karras , Orazio Gallo , Jonathan Tremblay
Abstract: A single two-dimensional (2D) image can be used as input to obtain a three-dimensional (3D) representation of the 2D image. This is done by extracting features from the 2D image by an encoder and determining a 3D representation of the 2D image utilizing a trained 2D convolutional neural network (CNN). Volumetric rendering is then run on the 3D representation to combine features within one or more viewing directions, and the combined features are provided as input to a multilayer perceptron (MLP) that predicts and outputs color (or multi-dimensional neural features) and density values for each point within the 3D representation. As a result, single-image inverse rendering may be performed using only a single 2D image as input to create a corresponding 3D representation of the scene in the single 2D image.
-
10.发明公开公开(公告)号：US20240104842A1
公开(公告)日：2024-03-28
申请号：US18472653
申请日：2023-09-22
Applicant: NVIDIA Corporation
Inventor： Koki Nagano , Alexander Trevithick , Chao Liu , Eric Ryan Chan , Sameh Khamis , Michael Stengel , Zhiding Yu
IPC: G06T17/00 , G06T5/20 , G06T7/70 , G06T7/90 , G06V10/771
CPC classification number: G06T17/00 , G06T5/20 , G06T7/70 , G06T7/90 , G06V10/771 , G06T2207/10024
Abstract: A method for generating, by an encoder-based model, a three-dimensional (3D) representation of a two-dimensional (2D) image is provided. The encoder-based model is trained to infer the 3D representation using a synthetic training data set generated by a pre-trained model. The pre-trained model is a 3D generative model that produces a 3D representation and a corresponding 2D rendering, which can be used to train a separate encoder-based model for downstream tasks like estimating a triplane representation, neural radiance field, mesh, depth map, 3D key points, or the like, given a single input image, using the pseudo ground truth 3D synthetic training data set. In a particular embodiment, the encoder-based model is trained to predict a triplane representation of the input image, which can then be rendered by a volume renderer according to pose information to generate an output image of the 3D scene from the corresponding viewpoint.
-
-
-
-
-
-
-
-
-
Search Results
13
records
(took 0.016 seconds)
