Multiview videos are acquired of a scene with corresponding cameras arranged at poses, such that there is view overlap between any pair of cameras. V-frames are generated from the multiview videos. The V-frames are encoded using only spatial prediction. Then, the V-frames are inserted periodically in an encoded bit stream to provide random temporal access to the multiview videos. Additional view dependency information enables the decoding of a reduced number of frames prior to accessing randomly a target frame for a specified view and time, and decoding the target frame. The method also decodes multiview videos by maintaining a reference picture list for a current frame of a plurality of multiview videos, and predicting each current frame of the plurality of multiview videos according to reference pictures indexed by the associated reference picture list.