A computer-implemented method is provided for processing a video stream in the compressed domain for watermarking, scrambling and other applications. Syntax elements are generated for input video as part of a video compression process. The syntax elements are entropy coded with an arithmetic entropy encoding process to produce a compressed bitstream for the input video. Regions of frames and related syntax elements of the input video are identified as candidates for modification. Based on metadata associated with a particular user, the syntax elements, the regions, and entropy coding state of the arithmetic entropy encoding process, bytes of the input video are changed to generate a modifying bitstream that is unique to the particular user; and modifying the compressed bitstream using the modifying bitstream to produce a decodable bitstream for the input video.