A method, medium, and apparatus for encoding and/or decoding video by generating a scalable bitstream supporting at least two bit-depths with forward compatibility, wherein the scalable bitstream comprises a base layer bitstream including a base quantization level and a base quantization parameter corresponding to a base bit-depth, and an enhancement layer bitstream including residue between an extended quantization level and a compensated quantization level that is predicted from the base quantization level, and a refined quantization parameter for refining the difference between an extended bit-depth and the base bit-depth.