A method and apparatus for a multiple-channel image/video coding system are disclosed. A residue generation process is applied to the image/video data to generate residue data. A set of integer operations is applied to the residue data across the input channels to generate residue transformed data having multiple output channels. In one embodiment, the residue transformed data associated with a first output channel is related to the difference between a first residue data associated with a first input channel and a second residue data associated with a second input channel. In another embodiment, the residue transformed data associated with a second output channel is related to the second difference between a threshold and a third residue data associated with a third input channel, and wherein the threshold corresponds to the first truncated or rounded average of the first residue data and the second residue data.