A method and apparatus for sample adaptive offset (SAO) compensation of reconstructed video data are disclosed. In one embodiment, the relation between the current pixel and said one or more neighboring pixels is stored so that the SAO compensated current pixel can replace the current pixel without buffering the to-be-processed pixels for classification. The SAO process may be performed on a region by region basis to adapt to the local characteristics of the picture.