-
公开(公告)号:US20240404174A1
公开(公告)日:2024-12-05
申请号:US18653723
申请日:2024-05-02
Applicant: NVIDIA Corporation
Inventor: Xueting Li , Shalini De Mello , Sifei Liu , Koki Nagano , Umar Iqbal , Jan Kautz
Abstract: Systems and methods are disclosed that animate a source portrait image with motion (i.e., pose and expression) from a target image. In contrast to conventional systems, given an unseen single-view portrait image, an implicit three-dimensional (3D) head avatar is constructed that not only captures photo-realistic details within and beyond the face region, but also is readily available for animation without requiring further optimization during inference. In an embodiment, three processing branches of a system produce three tri-planes representing coarse 3D geometry for the head avatar, detailed appearance of a source image, as well as the expression of a target image. By applying volumetric rendering to a combination of the three tri-planes, an image of the desired identity, expression and pose is generated.
-
公开(公告)号:US20240070874A1
公开(公告)日:2024-02-29
申请号:US18135654
申请日:2023-04-17
Applicant: NVIDIA Corporation
Inventor: Muhammed Kocabas , Ye Yuan , Umar Iqbal , Pavlo Molchanov , Jan Kautz
CPC classification number: G06T7/20 , G06T7/70 , G06T2207/20084 , G06T2207/30196 , G06T2207/30252 , G06T2210/12
Abstract: Estimating motion of a human or other object in video is a common computer task with applications in robotics, sports, mixed reality, etc. However, motion estimation becomes difficult when the camera capturing the video is moving, because the observed object and camera motions are entangled. The present disclosure provides for joint estimation of the motion of a camera and the motion of articulated objects captured in video by the camera.
-
公开(公告)号:US20230368501A1
公开(公告)日:2023-11-16
申请号:US18114177
申请日:2023-02-24
Applicant: NVIDIA Corporation
Inventor: Seonwook Park , Shalini De Mello , Pavlo Molchanov , Umar Iqbal , Jan Kautz
IPC: G06V10/772 , G06F7/57 , G06F17/18 , G06N3/088 , G06N3/045 , G06N3/047 , G06V10/774 , G06V10/82
CPC classification number: G06V10/772 , G06F7/57 , G06F17/18 , G06N3/088 , G06N3/045 , G06N3/047 , G06V10/774 , G06V10/82
Abstract: A neural network is trained to identify one or more features of an image. The neural network is trained using a small number of original images, from which a plurality of additional images are derived. The additional images generated by rotating and decoding embeddings of the image in a latent space generated by an autoencoder. The images generated by the rotation and decoding exhibit changes to a feature that is in proportion to the amount of rotation.
-
14.
公开(公告)号:US20210150757A1
公开(公告)日:2021-05-20
申请号:US16690015
申请日:2019-11-20
Applicant: NVIDIA Corporation
Inventor: Siva Karthik Mustikovela , Varun Jampani , Shalini De Mello , Sifei Liu , Umar Iqbal , Jan Kautz
Abstract: Apparatuses, systems, and techniques to identify orientations of objects within images. In at least one embodiment, one or more neural networks are trained to identify an orientations of one or more objects based, at least in part, on one or more characteristics of the object other than the object's orientation.
-
公开(公告)号:US12100113B2
公开(公告)日:2024-09-24
申请号:US17584213
申请日:2022-01-25
Applicant: NVIDIA Corporation
Inventor: Ye Yuan , Umar Iqbal , Pavlo Molchanov , Jan Kautz
CPC classification number: G06T19/20 , G06T7/0002 , G06T7/20 , G06T2207/10016 , G06T2207/20084 , G06T2207/30241 , G06T2219/2016
Abstract: In order to determine accurate three-dimensional (3D) models for objects within a video, the objects are first identified and tracked within the video, and a pose and shape are estimated for these tracked objects. A translation and global orientation are removed from the tracked objects to determine local motion for the objects, and motion infilling is performed to fill in any missing portions for the object within the video. A global trajectory is then determined for the objects within the video, and the infilled motion and global trajectory are then used to determine infilled global motion for the object within the video. This enables the accurate depiction of each object as a 3D pose sequence for that model that accounts for occlusions and global factors within the video.
-
公开(公告)号:US20240169636A1
公开(公告)日:2024-05-23
申请号:US18317378
申请日:2023-05-15
Applicant: NVIDIA Corporation
Inventor: Ye Yuan , Jiaming Song , Umar Iqbal , Arash Vahdat , Jan Kautz
CPC classification number: G06T13/40 , G06T5/002 , G06T13/80 , G06T2207/20081 , G06T2207/20084
Abstract: Systems and methods are disclosed that improve performance of synthesized motion generated by a diffusion neural network model. A physics-guided motion diffusion model incorporates physical constraints into the diffusion process to model the complex dynamics induced by forces and contact. Specifically, a physics-based motion projection module uses motion imitation in a physics simulator to project the denoised motion of a diffusion step to a physically plausible motion. The projected motion is further used in the next diffusion iteration to guide the denoising diffusion process. The use of physical constraints in the physics-guided motion diffusion model iteratively pulls the motion toward a physically-plausible space, reducing artifacts such as floating, foot sliding, and ground penetration.
-
公开(公告)号:US20230137403A1
公开(公告)日:2023-05-04
申请号:US17514730
申请日:2021-10-29
Applicant: Nvidia Corporation
Inventor: Orazio Gallo , Umar Iqbal , Atsuhiro Noguchi
IPC: H04N13/282 , G06N3/08 , G06K9/00 , H04N13/275 , H04N13/161
Abstract: Apparatuses, systems, and techniques are presented to generate one or more images. In at least one embodiment, one or more neural networks are used to generate one or more images of one or more objects in two or more different poses from two or more different points of view.
-
公开(公告)号:US20230070514A1
公开(公告)日:2023-03-09
申请号:US17584213
申请日:2022-01-25
Applicant: NVIDIA Corporation
Inventor: Ye Yuan , Umar Iqbal , Pavlo Molchanov , Jan Kautz
Abstract: In order to determine accurate three-dimensional (3D) models for objects within a video, the objects are first identified and tracked within the video, and a pose and shape are estimated for these tracked objects. A translation and global orientation are removed from the tracked objects to determine local motion for the objects, and motion infilling is performed to fill in any missing portions for the object within the video. A global trajectory is then determined for the objects within the video, and the infilled motion and global trajectory are then used to determine infilled global motion for the object within the video. This enables the accurate depiction of each object as a 3D pose sequence for that model that accounts for occlusions and global factors within the video.
-
公开(公告)号:US11593661B2
公开(公告)日:2023-02-28
申请号:US16389832
申请日:2019-04-19
Applicant: NVIDIA Corporation
Inventor: Seonwook Park , Shalini De Mello , Pavlo Molchanov , Umar Iqbal , Jan Kautz
Abstract: A neural network is trained to identify one or more features of an image. The neural network is trained using a small number of original images, from which a plurality of additional images are derived. The additional images generated by rotating and decoding embeddings of the image in a latent space generated by an autoencoder. The images generated by the rotation and decoding exhibit changes to a feature that is in proportion to the amount of rotation.
-
公开(公告)号:US20230004760A1
公开(公告)日:2023-01-05
申请号:US17361202
申请日:2021-06-28
Applicant: NVIDIA Corporation
Inventor: Siva Karthik Mustikovela , Shalini De Mello , Aayush Prakash , Umar Iqbal , Sifei Liu , Jan Kautz
IPC: G06K9/62
Abstract: Apparatuses, systems, and techniques to identify objects within an image using self-supervised machine learning. In at least one embodiment, a machine learning system is trained to recognize objects by training a first network to recognize objects within images that are generated by a second network. In at least one embodiment, the second network is a controllable network.
-
-
-
-
-
-
-
-
-