A method and apparatus for the scene-based watermarking of video data is disclosed. In one embodiment, each of a number of frames of a scene of video host data undergoes a temporal wavelet transform, from which blocks are extracted. The blocks undergo perceptual masking in the frequency domain, such that a watermark is embedded therein. Once the watermark block is taken out of the frequency domain, a spatial mask of the original block is weighted to the watermark block, and added to the original block to obtain the watermarked block.