An apparatus for sample adaptive offset (SAO) filtering in video encoding. A unified processing engine collects statistics on a block of pixels, determines a minimum RD cost (J) for each category of band offsets and edge offsets; determines a RD cost to find the optimal SAO type and determines a cost for each of the left SAO parameters and the up SAO parameters. The unified processing engine operates for three iterations: once for luminance once for each chrominance. A SAO merge decision unit determines an optimal mode and generates current LCU Parameters. The RD offset unit determination includes determining whether the sign of the minimum offset is proper for the category of edge offset. The RD offset is determined using a programmable look-up table indexed by the offset to estimate a rate. The unified processing engine operates on a three stage pipeline: loading blocks; processing; and updating blocks.