摘要:
A system and process for generating, and then rendering and displaying, an interactive viewpoint video in which a user can watch a dynamic scene while manipulating (freezing, slowing down, or reversing) time and changing the viewpoint at will. In general, the interactive viewpoint video is generated using a small number of cameras to capture multiple video streams. A multi-view 3D reconstruction and matting technique is employed to create a layered representation of the video frames that enables both efficient compression and interactive playback of the captured dynamic scene, while at the same time allowing for real-time rendering.
摘要:
A system and process for rendering and displaying an interactive viewpoint video is presented in which a user can watch a dynamic scene while manipulating (freezing, slowing down, or reversing) time and changing the viewpoint at will. The ability to interactively control viewpoint while watching a video is an exciting new application for image-based rendering. Because any intermediate view can be synthesized at any time, with the potential for space-time manipulation, this type of video has been dubbed interactive viewpoint video.
摘要:
A system and method for deghosting mosaics provides a novel multiperspective plane sweep approach for generating an image mosaic from a sequence of still images, video images, scanned photographic images, computer generated images, etc. This multiperspective plane sweep approach uses virtual camera positions to compute depth maps for columns of overlapping pixels in adjacent images. Object distortions and ghosting caused by image parallax when generating the image mosaics are then minimized by blending pixel colors, or grey values, for each computed depth to create a common composite area for each of the overlapping images. Further, the multiperspective plane sweep approach described herein is both computationally efficient, and applicable to both the case of limited overlap between the images used for creating the image mosaics, and to the case of extensive or increased image overlap.
摘要:
A system and process for generating a two-layer, 3D representation of a digital or digitized image from the image and a pixel disparity map of the image is presented. The two layer representation includes a main layer having pixels exhibiting background colors and background disparities associated with correspondingly located pixels of depth discontinuity areas in the image, as well as pixels exhibiting colors and disparities associated with correspondingly located pixels of the image not found in these depth discontinuity areas. The other layer is a boundary layer made up of pixels exhibiting foreground colors, foreground disparities and alpha values associated with the correspondingly located pixels of the depth discontinuity areas. The depth discontinuity areas correspond to prescribed sized areas surrounding depth discontinuities found in the image using a disparity map thereof.
摘要:
A system and process for providing an interactive video tour of a tour site to a user is presented. In general, the system and process provides an image-based rendering system that enables users to explore remote real world locations, such as a house or a garden. The present approach is based directly on filming an environment, and then using image-based rendering techniques to replay the tour in an interactive manner. As such, the resulting experience is referred to as Interactive Video Tours. The experience is interactive in that the user can move freely along a path, choose between different directions of motion at branch points in the path, and look around in any direction. The user experience is additionally enhanced with multimedia elements such as overview maps, video textures, and sound.
摘要:
Methods and systems for generating a depth map are provided. The method includes projecting an infrared (IR) dot pattern onto a scene. The method also includes capturing stereo images from each of two or more synchronized IR cameras, detecting a number of dots within the stereo images, computing a number of feature descriptors for the dots in the stereo images, and computing a disparity map between the stereo images. The method further includes generating a depth map for the scene using the disparity map.
摘要:
Two-dimensional (2D) video is converted into multi-view video. The 2D video is segmented to generate a temporally consistent segmented 2D video which is made up of a sequence of segmented frames. The multi-view video is generated by employing user-guided operations to generate depth assignments for the segments associated with user-assigned regions of the segmented frames, where a user-assigned region is formed from a group of contiguous segments selected by the user.
摘要:
Deinterlacing of video involves converting interlaced video to progressive video by interpolating a missing pixel in the interlaced video from other pixels in the video. A plurality of interpolants are provided, each of which interpolates a pixel value from other pixels that are nearby in space and/or time. The data costs of using the various interpolants is calculated. A particular one of the interpolants is chosen based on the data costs associated with the various interpolants. The chosen interpolant is used to interpolate the value of the missing pixel. The interpolated pixel value may be refined based on exemplars. The exemplars may be taken from the video that is being deinterlaced.
摘要:
Matte-based video restoration technique embodiments are presented which model spatio-temporally varying film wear artifacts found in digitized copies of film media. In general, this is accomplished by employing residual color information in recovering of artifact mattes. To this end, the distributions of artifact colors and their fractional contribution to each pixel of each frame being considered are extracted based on color information from the spatial and temporal neighborhoods of the pixel. The extracted information can then be used to restore the video by removing the artifacts.
摘要:
Deinterlacing of video involves converting interlaced video to progressive video by interpolating a missing pixel in the interlaced video from other pixels in the video. A plurality of interpolants are provided, each of which interpolates a pixel value from other pixels that are nearby in space and/or time. The data costs of using the various interpolants is calculated. A particular one of the interpolants is chosen based on the data costs associated with the various interpolants. The chosen interpolant is used to interpolate the value of the missing pixel. The interpolated pixel value may be refined based on exemplars. The exemplars may be taken from the video that is being deinterlaced.