A scalable video decoder and a controlling method thereof are provided. A method of controlling a scalable video decoder includes: determining whether a macro block mode of an upper layer is an interlayer prediction mode when decoding a scalable video; when the macro block mode of the upper layer is determined as the interlayer prediction mode, determining an up-sampling operation is required to thereby either perform texture up-sampling using corresponding macro block information of a lower layer and neighboring pixel information, or perform residual up-sampling using the corresponding macro block information of the lower layer; and decoding the upper layer using the texture up-sampling or the residual up-sampling.