An example device for decoding video data includes a memory configured to store video data and one or more processors implemented in circuitry and configured to determine a maximum possible value for a secondary transform syntax element for a block of video data, entropy decode a value for the secondary transform syntax element of the block to form a binarized value representative of the secondary transform for the block, reverse binarize the value for the secondary transform syntax element using a common binarization scheme regardless of the maximum possible value to determine the secondary transform for the block, and inverse-transform transform coefficients of the block using the determined secondary transform.