-
公开(公告)号:US20180288431A1
公开(公告)日:2018-10-04
申请号:US15939098
申请日:2018-03-28
Applicant: NVIDIA Corporation
Inventor: Ming-Yu Liu , Xiaodong Yang , Jan Kautz , Sergey Tulyakov
IPC: H04N19/513 , G06K9/00 , G06N3/08 , G06T13/40
CPC classification number: H04N19/521 , G06K9/00201 , G06K9/00281 , G06N3/0445 , G06N3/0454 , G06N3/0472 , G06N3/08 , G06T13/40 , G06T2207/20081 , G06T2207/30196
Abstract: A method, computer readable medium, and system are disclosed for action video generation. The method includes the steps of generating, by a recurrent neural network, a sequence of motion vectors from a first set of random variables and receiving, by a generator neural network, the sequence of motion vectors and a content vector sample. The sequence of motion vectors and the content vector sample are sampled by the generator neural network to produce a video clip.
-
公开(公告)号:US20180247201A1
公开(公告)日:2018-08-30
申请号:US15907098
申请日:2018-02-27
Applicant: NVIDIA Corporation
Inventor: Ming-Yu Liu , Thomas Michael Breuel , Jan Kautz
CPC classification number: G06N3/088 , G06N3/0454 , G06N3/0472 , G06N3/063 , G06N3/084 , G06T1/00 , G06T3/4046
Abstract: A method, computer readable medium, and system are disclosed for training a neural network. The method includes the steps of encoding, by a first neural network, a first image represented in a first domain to convert the first image to a shared latent space, producing a first latent code and encoding, by a second neural network, a second image represented in a second domain to convert the second image to a shared latent space, producing a second latent code. The method also includes the step of generating, by a third neural network, a first translated image in the second domain based on the first latent code, wherein the first translated image is correlated with the first image and weight values of the third neural network are computed based on the first latent code and the second latent code.
-
公开(公告)号:US20180181809A1
公开(公告)日:2018-06-28
申请号:US15855887
申请日:2017-12-27
Applicant: NVIDIA Corporation
Inventor: Rajeev Ranjan , Shalini De Mello , Jan Kautz
CPC classification number: G06K9/00604 , G06K9/00228 , G06K9/00255 , G06K9/00617 , G06K9/00973 , G06K9/00986 , G06K9/3216 , G06K9/4628 , G06K9/6256 , G06K9/627 , G06N3/04 , G06N3/0454 , G06N3/08 , G06T7/11 , G06T7/70 , G06T7/73 , G06T2200/04 , G06T2207/10024 , G06T2207/20081 , G06T2207/20084 , G06T2207/30201 , G06T2210/52
Abstract: A method, computer readable medium, and system are disclosed for performing unconstrained appearance-based gaze estimation. The method includes the steps of identifying an image of an eye and a head orientation associated with the image of the eye, determining an orientation for the eye by analyzing, within a convolutional neural network (CNN), the image of the eye and the head orientation associated with the image of the eye, and returning the orientation of the eye.
-
64.
公开(公告)号:US20170206405A1
公开(公告)日:2017-07-20
申请号:US15402128
申请日:2017-01-09
Applicant: NVIDIA Corporation
Inventor: Pavlo Molchanov , Xiaodong Yang , Shalini De Mello , Kihwan Kim , Stephen Walter Tyree , Jan Kautz
CPC classification number: G06K9/00355 , G06K9/00201 , G06K9/00765 , G06K9/4628 , G06K9/4652 , G06K9/6251 , G06K9/6256 , G06K9/627 , G06K9/6277 , G06N3/0445 , G06N3/0454 , G06N3/084 , Y04S10/54
Abstract: A method, computer readable medium, and system are disclosed for detecting and classifying hand gestures. The method includes the steps of receiving an unsegmented stream of data associated with a hand gesture, extracting spatio-temporal features from the unsegmented stream by a three-dimensional convolutional neural network (3DCNN), and producing a class label for the hand gesture based on the spatio-temporal features.
-
公开(公告)号:US12100113B2
公开(公告)日:2024-09-24
申请号:US17584213
申请日:2022-01-25
Applicant: NVIDIA Corporation
Inventor: Ye Yuan , Umar Iqbal , Pavlo Molchanov , Jan Kautz
CPC classification number: G06T19/20 , G06T7/0002 , G06T7/20 , G06T2207/10016 , G06T2207/20084 , G06T2207/30241 , G06T2219/2016
Abstract: In order to determine accurate three-dimensional (3D) models for objects within a video, the objects are first identified and tracked within the video, and a pose and shape are estimated for these tracked objects. A translation and global orientation are removed from the tracked objects to determine local motion for the objects, and motion infilling is performed to fill in any missing portions for the object within the video. A global trajectory is then determined for the objects within the video, and the infilled motion and global trajectory are then used to determine infilled global motion for the object within the video. This enables the accurate depiction of each object as a 3D pose sequence for that model that accounts for occlusions and global factors within the video.
-
公开(公告)号:US20240169636A1
公开(公告)日:2024-05-23
申请号:US18317378
申请日:2023-05-15
Applicant: NVIDIA Corporation
Inventor: Ye Yuan , Jiaming Song , Umar Iqbal , Arash Vahdat , Jan Kautz
CPC classification number: G06T13/40 , G06T5/002 , G06T13/80 , G06T2207/20081 , G06T2207/20084
Abstract: Systems and methods are disclosed that improve performance of synthesized motion generated by a diffusion neural network model. A physics-guided motion diffusion model incorporates physical constraints into the diffusion process to model the complex dynamics induced by forces and contact. Specifically, a physics-based motion projection module uses motion imitation in a physics simulator to project the denoised motion of a diffusion step to a physically plausible motion. The projected motion is further used in the next diffusion iteration to guide the denoising diffusion process. The use of physical constraints in the physics-guided motion diffusion model iteratively pulls the motion toward a physically-plausible space, reducing artifacts such as floating, foot sliding, and ground penetration.
-
公开(公告)号:US11989642B2
公开(公告)日:2024-05-21
申请号:US17952866
申请日:2022-09-26
Applicant: NVIDIA Corporation
Inventor: Ruben Villegas , Alejandro Troccoli , Iuri Frosio , Stephen Tyree , Wonmin Byeon , Jan Kautz
Abstract: In various examples, historical trajectory information of objects in an environment may be tracked by an ego-vehicle and encoded into a state feature. The encoded state features for each of the objects observed by the ego-vehicle may be used—e.g., by a bi-directional long short-term memory (LSTM) network—to encode a spatial feature. The encoded spatial feature and the encoded state feature for an object may be used to predict lateral and/or longitudinal maneuvers for the object, and the combination of this information may be used to determine future locations of the object. The future locations may be used by the ego-vehicle to determine a path through the environment, or may be used by a simulation system to control virtual objects—according to trajectories determined from the future locations—through a simulation environment.
-
公开(公告)号:US20240070987A1
公开(公告)日:2024-02-29
申请号:US18110287
申请日:2023-02-15
Applicant: NVIDIA Corporation
Inventor: Xueting Li , Sifei Liu , Shalini De Mello , Orazio Gallo , Jiashun Wang , Jan Kautz
Abstract: Transferring pose to three-dimensional characters is a common computer graphics task that typically involves transferring the pose of a reference avatar to a (stylized) three-dimensional character. Since three-dimensional characters are created by professional artists through imagination and exaggeration, and therefore, unlike human or animal avatars, have distinct shape and features, matching the pose of a three-dimensional character to that of a reference avatar generally requires manually creating shape information for the three-dimensional character that is required for pose transfer. The present disclosure provides for the automated transfer of a reference pose to a three-dimensional character, based specifically on a learned shape code for the three-dimensional character.
-
公开(公告)号:US11880927B2
公开(公告)日:2024-01-23
申请号:US18320446
申请日:2023-05-19
Applicant: NVIDIA Corporation
Inventor: Xueting Li , Sifei Liu , Kihwan Kim , Shalini De Mello , Jan Kautz
CPC classification number: G06T15/04 , G06T7/579 , G06T7/70 , G06T15/20 , G06T17/20 , G06T2207/10016 , G06T2207/20084 , G06T2207/30244
Abstract: A three-dimensional (3D) object reconstruction neural network system learns to predict a 3D shape representation of an object from a video that includes the object. The 3D reconstruction technique may be used for content creation, such as generation of 3D characters for games, movies, and 3D printing. When 3D characters are generated from video, the content may also include motion of the character, as predicted based on the video. The 3D object construction technique exploits temporal consistency to reconstruct a dynamic 3D representation of the object from an unlabeled video. Specifically, an object in a video has a consistent shape and consistent texture across multiple frames. Texture, base shape, and part correspondence invariance constraints may be applied to fine-tune the neural network system. The reconstruction technique generalizes well—particularly for non-rigid objects.
-
公开(公告)号:US20240020897A1
公开(公告)日:2024-01-18
申请号:US17862818
申请日:2022-07-12
Applicant: Nvidia Corporation
Inventor: Ting-Chun Wang , Ming-Yu Liu , Koki Nagano , Sameh Khamis , Jan Kautz
CPC classification number: G06T11/60 , G06T5/006 , G06T2207/20084
Abstract: Apparatuses, systems, and techniques are presented to generate image data. In at least one embodiment, one or more neural networks are used to cause a lighting effect to be applied to one or more objects within one or more images based, at least in part, on synthetically generated images of the one or more objects.
-
-
-
-
-
-
-
-
-