-
公开(公告)号:US11496773B2
公开(公告)日:2022-11-08
申请号:US17352064
申请日:2021-06-18
Applicant: NVIDIA Corporation
Inventor: Yi-Hsuan Tsai , Ming-Yu Liu , Deqing Sun , Ming-Hsuan Yang , Jan Kautz
IPC: H04N19/85 , H04N19/91 , H04N19/436 , H04N19/46
Abstract: A method, computer readable medium, and system are disclosed for identifying residual video data. This data describes data that is lost during a compression of original video data. For example, the original video data may be compressed and then decompressed, and this result may be compared to the original video data to determine the residual video data. This residual video data is transformed into a smaller format by means of encoding, binarizing, and compressing, and is sent to a destination. At the destination, the residual video data is transformed back into its original format and is used during the decompression of the compressed original video data to improve a quality of the decompressed original video data.
-
公开(公告)号:US11082720B2
公开(公告)日:2021-08-03
申请号:US16191174
申请日:2018-11-14
Applicant: NVIDIA Corporation
Inventor: Yi-Hsuan Tsai , Ming-Yu Liu , Deqing Sun , Ming-Hsuan Yang , Jan Kautz
IPC: H04N19/85 , H04N19/91 , H04N19/436 , H04N19/46
Abstract: A method, computer readable medium, and system are disclosed for identifying residual video data. This data describes data that is lost during a compression of original video data. For example, the original video data may be compressed and then decompressed, and this result may be compared to the original video data to determine the residual video data. This residual video data is transformed into a smaller format by means of encoding, binarizing, and compressing, and is sent to a destination. At the destination, the residual video data is transformed back into its original format and is used during the decompression of the compressed original video data to improve a quality of the decompressed original video data.
-
3.
公开(公告)号:US20200334502A1
公开(公告)日:2020-10-22
申请号:US16921012
申请日:2020-07-06
Applicant: NVIDIA Corporation
Inventor: Wei-Chih Tu , Ming-Yu Liu , Varun Jampani , Deqing Sun , Ming-Hsuan Yang , Jan Kautz
Abstract: Segmentation is the identification of separate objects within an image. An example is identification of a pedestrian passing in front of a car, where the pedestrian is a first object and the car is a second object. Superpixel segmentation is the identification of regions of pixels within an object that have similar properties. An example is identification of pixel regions having a similar color, such as different articles of clothing worn by the pedestrian and different components of the car. A pixel affinity neural network (PAN) model is trained to generate pixel affinity maps for superpixel segmentation. The pixel affinity map defines the similarity of two points in space. In an embodiment, the pixel affinity map indicates a horizontal affinity and vertical affinity for each pixel in the image. The pixel affinity map is processed to identify the superpixels.
-
公开(公告)号:US10748036B2
公开(公告)日:2020-08-18
申请号:US16188641
申请日:2018-11-13
Applicant: NVIDIA Corporation
Inventor: Wei-Chih Tu , Ming-Yu Liu , Varun Jampani , Deqing Sun , Ming-Hsuan Yang , Jan Kautz
Abstract: Segmentation is the identification of separate objects within an image. An example is identification of a pedestrian passing in front of a car, where the pedestrian is a first object and the car is a second object. Superpixel segmentation is the identification of regions of pixels within an object that have similar properties An example is identification of pixel regions having a similar color, such as different articles of clothing worn by the pedestrian and different components of the car. A pixel affinity neural network (PAN) model is trained to generate pixel affinity maps for superpixel segmentation. The pixel affinity map defines the similarity of two points in space. In an embodiment, the pixel affinity map indicates a horizontal affinity and vertical affinity for each pixel in the image. The pixel affinity map is processed to identify the superpixels.
-
公开(公告)号:US20180293737A1
公开(公告)日:2018-10-11
申请号:US15942213
申请日:2018-03-30
Applicant: NVIDIA Corporation
Inventor: Deqing Sun , Xiaodong Yang , Ming-Yu Liu , Jan Kautz
CPC classification number: G06T7/207 , G06N3/0454 , G06N3/08 , G06N5/046 , G06T3/0093 , G06T7/246 , G06T7/251 , G06T7/97 , G06T2200/28 , G06T2207/10016 , G06T2207/20016 , G06T2207/20032 , G06T2207/20084
Abstract: A method, computer readable medium, and system are disclosed for estimating optical flow between two images. A first pyramidal set of features is generated for a first image and a partial cost volume for a level of the first pyramidal set of features is computed, by a neural network, using features at the level of the first pyramidal set of features and warped features extracted from a second image, where the partial cost volume is computed across a limited range of pixels that is less than a full resolution of the first image, in pixels, at the level. The neural network processes the features and the partial cost volume to produce a refined optical flow estimate for the first image and the second image.
-
公开(公告)号:US11508076B2
公开(公告)日:2022-11-22
申请号:US17156406
申请日:2021-01-22
Applicant: NVIDIA Corporation
Inventor: Zhaoyang Lv , Kihwan Kim , Deqing Sun , Alejandro Jose Troccoli , Jan Kautz
IPC: G06T7/254 , G06T7/90 , G06T7/50 , G06N3/08 , G06T7/194 , G06T3/00 , G06T7/70 , G06T7/60 , G06T7/11 , G06N5/04 , G06T7/285 , G06T7/215
Abstract: A neural network model receives color data for a sequence of images corresponding to a dynamic scene in three-dimensional (3D) space. Motion of objects in the image sequence results from a combination of a dynamic camera orientation and motion or a change in the shape of an object in the 3D space. The neural network model generates two components that are used to produce a 3D motion field representing the dynamic (non-rigid) part of the scene. The two components are information identifying dynamic and static portions of each image and the camera orientation. The dynamic portions of each image contain motion in the 3D space that is independent of the camera orientation. In other words, the motion in the 3D space (estimated 3D scene flow data) is separated from the motion of the camera.
-
公开(公告)号:US20210067735A1
公开(公告)日:2021-03-04
申请号:US16559312
申请日:2019-09-03
Applicant: Nvidia Corporation
Inventor: Fitsum Reda , Deqing Sun , Aysegul Dundar , Mohammad Shoeybi , Guilin Liu , Kevin Shih , Andrew Tao , Jan Kautz , Bryan Catanzaro
Abstract: Apparatuses, systems, and techniques to enhance video. In at least one embodiment, one or more neural networks are used to create, from a first video, a second video having a higher frame rate, higher resolution, or reduced number of missing or corrupt video frames.
-
公开(公告)号:US20200084427A1
公开(公告)日:2020-03-12
申请号:US16569104
申请日:2019-09-12
Applicant: NVIDIA Corporation
Inventor: Deqing Sun , Varun Jampani , Erik Gundersen Learned-Miller , Huaizu Jiang
IPC: H04N13/122 , H04N13/128 , G06N3/08
Abstract: Scene flow represents the three-dimensional (3D) structure and movement of objects in a video sequence in three dimensions from frame-to-frame and is used to track objects and estimate speeds for autonomous driving applications. Scene flow is recovered by a neural network system from a video sequence captured from at least two viewpoints (e.g., cameras), such as a left-eye and right-eye of a viewer. An encoder portion of the system extracts features from frames of the video sequence. The features are input to a first decoder to predict optical flow and a second decoder to predict disparity. The optical flow represents pixel movement in (x,y) and the disparity represents pixel movement in z (depth). When combined, the optical flow and disparity represent the scene flow.
-
公开(公告)号:US10424069B2
公开(公告)日:2019-09-24
申请号:US15942213
申请日:2018-03-30
Applicant: NVIDIA Corporation
Inventor: Deqing Sun , Xiaodong Yang , Ming-Yu Liu , Jan Kautz
Abstract: A method, computer readable medium, and system are disclosed for estimating optical flow between two images. A first pyramidal set of features is generated for a first image and a partial cost volume for a level of the first pyramidal set of features is computed, by a neural network, using features at the level of the first pyramidal set of features and warped features extracted from a second image, where the partial cost volume is computed across a limited range of pixels that is less than a full resolution of the first image, in pixels, at the level. The neural network processes the features and the partial cost volume to produce a refined optical flow estimate for the first image and the second image.
-
公开(公告)号:US20210150736A1
公开(公告)日:2021-05-20
申请号:US17156406
申请日:2021-01-22
Applicant: NVIDIA Corporation
Inventor: Zhaoyang Lv , Kihwan Kim , Deqing Sun , Alejandro Jose Troccoli , Jan Kautz
IPC: G06T7/254 , G06T7/90 , G06T7/50 , G06N3/08 , G06T7/194 , G06T3/00 , G06T7/70 , G06T7/60 , G06T7/11 , G06N5/04 , G06T7/285 , G06T7/215
Abstract: A neural network model receives color data for a sequence of images corresponding to a dynamic scene in three-dimensional (3D) space. Motion of objects in the image sequence results from a combination of a dynamic camera orientation and motion or a change in the shape of an object in the 3D space. The neural network model generates two components that are used to produce a 3D motion field representing the dynamic (non-rigid) part of the scene. The two components are information identifying dynamic and static portions of each image and the camera orientation. The dynamic portions of each image contain motion in the 3D space that is independent of the camera orientation. In other words, the motion in the 3D space (estimated 3D scene flow data) is separated from the motion of the camera.
-
-
-
-
-
-
-
-
-