Methods and apparatuses for performing inter-color-plane prediction with adaptability to various existing video content representations are provided. A plurality of predetermined rescaling schemes based on a color plane format is selected. A first block of original samples of a first color plane is encoded into a compressed bitstream. A block of reconstructed samples of the first color plane is reconstructed. An inter-color-plane prediction process is performed to produce samples of a second color plane. Said block of prediction samples of the second color plane is subtracted from a second block of original samples of the second color plane to produce a block of residual samples of the second color plane where the positions of the first block and the second block of original samples are aligned. Finally, the block of residual samples of the second color plane is encoded.