Efficient image compression for video data characterized by a non-neutral dominant white point is achieved by transforming the input video signal into a de-correlated video signal based on a color difference encoding transform, wherein the color difference encoding transform is adapted based on the dominant white point using an algorithm. The adapting algorithm is designed for optimizing low-entropy output when the white point is other than a neutral or equal-energy value. Decompression is handled conversely.