公开(公告)号：US20240185396A1

公开(公告)日：2024-06-06

申请号：US18222725

申请日：2023-07-17

Applicant: NVIDIA CORPORATION

Inventor： Ali Hatamizadeh ,  Jiaming Song ,  Jan Kautz ,  Arash Vahdat

IPC: G06T5/00 ,  G06T1/20 ,  G06T7/00

CPC classification number: G06T5/002 ,  G06T1/20 ,  G06T7/0002 ,  G06T2207/20081 ,  G06T2207/20182

Abstract: Apparatuses, systems, and techniques to generate images. In at least one embodiment, one or more machine learning models generate an output image based, at least in part, on calculating attention scores using time embeddings.

公开(公告)号：US20250045892A1

公开(公告)日：2025-02-06

申请号：US18593742

申请日：2024-03-01

Applicant: NVIDIA Corporation

Inventor： Morteza Mardani ,  Jiaming Song ,  Jan Kautz ,  Arash Vahdat

IPC: G06T7/00 ,  G06T3/40 ,  G06T5/70 ,  G06T5/73 ,  G06T5/77

Abstract: Diffusion models are machine learning algorithms that are uniquely trained to generate high-quality data from an input lower-quality data. For example, they can be trained in the image domain, for example, to perform specific image restoration tasks, such as inpainting (e.g. completing an incomplete image), deblurring (e.g. removing blurring from an image), and super-resolution (e.g. increasing a resolution of an image), or they can be trained to perform image rendering tasks, including 2D-to-3D image generation tasks. However, current approaches to training diffusion models only allow the models to be optimized for a specific task such that they will not achieve high-quality results when used for other tasks. The present disclosure provides a diffusion model that uses variational inferencing to approximate a distribution of data, which allows the diffusion model to universally solve different tasks without having to be re-trained specifically for each task.

公开(公告)号：US20240253217A1

公开(公告)日：2024-08-01

申请号：US18538248

申请日：2023-12-13

Applicant: NVIDIA Corporation

Inventor： Arash Vahdat ,  Hongxu Yin ,  Jan Kautz ,  Jiaming Song ,  Ming-Yu Liu ,  Morteza Mardani ,  Qinsheng Zhang

IPC: B25J9/16

CPC classification number: B25J9/163 ,  B25J9/1664 ,  B25J9/1697

Abstract: Apparatuses, systems, and techniques to calculate a combined loss value based on applying one or more loss functions to the plurality of samples generated by a diffusion model to update the samples to determine a synthesized motions of one or more objects.

公开(公告)号：US20240169636A1

公开(公告)日：2024-05-23

申请号：US18317378

申请日：2023-05-15

Applicant: NVIDIA Corporation

Inventor： Ye Yuan ,  Jiaming Song ,  Umar Iqbal ,  Arash Vahdat ,  Jan Kautz

IPC: G06T13/40 ,  G06T5/00 ,  G06T13/80

CPC classification number: G06T13/40 ,  G06T5/002 ,  G06T13/80 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: Systems and methods are disclosed that improve performance of synthesized motion generated by a diffusion neural network model. A physics-guided motion diffusion model incorporates physical constraints into the diffusion process to model the complex dynamics induced by forces and contact. Specifically, a physics-based motion projection module uses motion imitation in a physics simulator to project the denoised motion of a diffusion step to a physically plausible motion. The projected motion is further used in the next diffusion iteration to guide the denoising diffusion process. The use of physical constraints in the physics-guided motion diffusion model iteratively pulls the motion toward a physically-plausible space, reducing artifacts such as floating, foot sliding, and ground penetration.

公开(公告)号：US20240046422A1

公开(公告)日：2024-02-08

申请号：US18169545

申请日：2023-02-15

Applicant: NVIDIA Corporation

Inventor： Jiaming Song

IPC: G06T5/00 ,  G06T5/50

CPC classification number: G06T5/002 ,  G06T5/50 ,  G06T5/005 ,  G06T2207/20084 ,  G06T2207/20081 ,  G06T2207/20212

Abstract: A diffusion model is augmented with pseudoinverse guidance to restore data, removing artifacts and generating high-quality reconstructed data from limited, low-quality and/or noisy input data. The low-quality input data is denoised by a diffusion model and the denoised input data is combined with a guidance term to produce output data of higher-quality compared with the low-quality input data. The guidance term is a vector-Jacobian product that encourages consistency between the denoised input data and measurements after a pseudoinverse transformation. The denoising process may be applied in an iterative fashion to generate valid solutions to the inverse problem. The augmented diffusion model is a problem-agnostic (e.g., plug-and-play) denoiser that can restore data for a variety of tasks. Example image restoration tasks include denoising, JPEG denoising, deblurring, outpainting, inpainting, colorization, high-dynamic range, and super-resolution.