A method and apparatus for video data processing for video in YUV422 or YUV 444 formats are disclosed. In one embodiment, for a 2N×2N luma coding unit (CU) in YUV422 format, the transform process partitions residue data corresponding to the 2N×2N luma CU and the N×2N chroma CU into square luma and chroma transform units (TUs). The residue data associated with the luma and the chroma CUs are generated by applying prediction process to the luma CU and the chroma CU. The transform process is independent of prediction block size or prediction mode associated with the prediction process. In another embodiment, the prediction process splits the CU into two prediction blocks. Transform process is applied on the chroma residue data corresponding to the chroma CU to form one or more chroma TUs, wherein the transform process is dependent on CU size and prediction block size, or CU size and prediction mode.