A video encoder generates a bitstream that includes a residual quad tree (RQT) for a coding unit (CU). The CU is larger than a maximum-allowable transform unit (TU) size and the RQT includes a hierarchy of nodes. A root node of the RQT corresponds to the CU as a whole and leaf nodes of the RQT correspond to TUs of the CU. The root node is associated with a coded block flag (CBF) for a chroma component. The CBF for the chroma component indicates whether any of the TUs of the CU are associated with a significant coefficient block that is based on samples of the particular chroma component. A video decoder receives the bitstream and determines, based on the CBF, whether coefficient blocks associated with TUs that correspond to the leaf nodes include non-zero coefficients.