公开(公告)号：US10986325B2

公开(公告)日：2021-04-20

申请号：US16569104

申请日：2019-09-12

Applicant: NVIDIA Corporation

Inventor： Deqing Sun ,  Varun Jampani ,  Erik Gundersen Learned-Miller ,  Huaizu Jiang

IPC: H04N13/122 ,  H04N13/128 ,  G06N3/08 ,  H04N13/00

Abstract: Scene flow represents the three-dimensional (3D) structure and movement of objects in a video sequence in three dimensions from frame-to-frame and is used to track objects and estimate speeds for autonomous driving applications. Scene flow is recovered by a neural network system from a video sequence captured from at least two viewpoints (e.g., cameras), such as a left-eye and right-eye of a viewer. An encoder portion of the system extracts features from frames of the video sequence. The features are input to a first decoder to predict optical flow and a second decoder to predict disparity. The optical flow represents pixel movement in (x,y) and the disparity represents pixel movement in z (depth). When combined, the optical flow and disparity represent the scene flow.

公开(公告)号：US20200084427A1

公开(公告)日：2020-03-12

申请号：US16569104

申请日：2019-09-12

Applicant: NVIDIA Corporation

Inventor： Deqing Sun ,  Varun Jampani ,  Erik Gundersen Learned-Miller ,  Huaizu Jiang

IPC: H04N13/122 ,  H04N13/128 ,  G06N3/08

Abstract: Scene flow represents the three-dimensional (3D) structure and movement of objects in a video sequence in three dimensions from frame-to-frame and is used to track objects and estimate speeds for autonomous driving applications. Scene flow is recovered by a neural network system from a video sequence captured from at least two viewpoints (e.g., cameras), such as a left-eye and right-eye of a viewer. An encoder portion of the system extracts features from frames of the video sequence. The features are input to a first decoder to predict optical flow and a second decoder to predict disparity. The optical flow represents pixel movement in (x,y) and the disparity represents pixel movement in z (depth). When combined, the optical flow and disparity represent the scene flow.