Wavelet based multiresolution video representations generated by multi-scale motion compensated temporal filtering (MCTF) and spatial wavelet transform are disclosed. Since temporal filtering and spatial filtering are separated in generating such representations, there are many different ways to intertwine single-level MCTF and single-level spatial filtering, resulting in many different video representation schemes with spatially scalable motion vectors for the support of different combination of spatial scalability and temporal scalability. The problem of design of such a video representation scheme to full the spatial/temporal scalability requirements is studied. Signaling of the scheme to the decoder is also investigated. Since MCTF is performed subband by subband, motion vectors are available for reconstructing video sequences of any possible reduced spatial resolution, restricted by the dyadic decomposition pattern and the maximal spatial decomposition level. It is thus clear that the family of decomposition schemes provides efficient and versatile multiresolution video representations for fully scalable video coding.