A method and apparatus of video coding operate by receiving input data associated with a current data unit in a current picture, wherein the current data unit includes a luma component and a chroma component and the current data unit includes a luma data unit and a chroma data unit. The operation proceeds by splitting the luma data unit and the Chroma data unit using one shared tree until the luma data unit and the chroma data unit reach a stop node, encoding or decoding the stop node as a leaf CU (coding unit) if the stop node is greater than M×N for the luma component, M and N are positive integers, and signalling or parsing a prediction mode for the stop node if the stop node is smaller than or equal to M×N for the luma component.