A video coding device (e.g., a video encoder or a video decoder) is configured to perform various transformations on video data. The video coding device applies a primary transform to a block of the video data, the primary transform having a first size, and the sub-block being at least a portion of the block. The video coding device determines whether application of a secondary transform, having a second size, to a sub-block of the block is allowed. Application of the secondary transform is disallowed when the first size is equal to the second size. Based on the application of the secondary transform being allowed, the video coding device applies the secondary transform to the sub-block. Application of the primary transform and the secondary transform construct a residual block in a pixel domain.