Embodiments of the present invention disclose a system and method for multi-viewpoint video capture. According to one embodiment, the system includes a camera housing for accommodating both a first multi-imager set and a second multi-imager set, with each multi-imager set including a plurality of optical cameras having different viewpoint directions and configured to produce a source image. Furthermore, each camera in the first multi-imager set and the second multi-imager set include corresponding cameras facing in approximately the same viewpoint direction. The first multi-imager set is positioned laterally adjacent to the second multi-imager set such that lines joining a center of projection of corresponding cameras in the first multi-imager set and second multi-imager set are approximately parallel.