A video encoder generates a sequence of sample adaptive offset (SAO) syntax elements for a coding tree block. The SAO syntax elements include regular context-adaptive binary arithmetic coding (CABAC) coded bins for a color component and bypass-coded bins for the color component. None of the bypass-coded bins is between two of the regular CABAC-coded bins in the sequence. The video encoder uses regular CABAC to encode the regular CABAC-coded bins and uses bypass coding to encode the bypass-coded bins. The video encoder outputs the SAO syntax elements in a bitstream. A video decoder receives the bitstream, uses regular CABAC to decode the regular CABAC-coded bins, uses bypass coding to decode the bypass-coded bins, and modifies a reconstructed picture based on the SAO syntax elements.