A method of modified SAO (sample-adaptive offset) processing for a reconstructed picture in a video coding system to improve the performance is disclosed. In one example, a SAO-sign threshold is introduced to determine the sign of the difference between a current reconstructed pixel and a neighboring reconstructed pixel. A range of difference values greater than the negative SAO-sign threshold and smaller than the positive SAO-sign threshold is assigned to have a sign value of 0. In another example, the SAO-offset value is derived by multiplying the SAO-offset sign with a result from applying left shift by the SAO-bit-shift value to the absolute SAO-offset value. In yet another example, the absolute SAO-offset value is coded by truncated Rice (TR) codes and a maximum TR value is indicated by a syntax element.