Automatic gain control systems disclosed herein can incorporate a confidence metric that can estimate the accuracy of gain adjustments calculated by an automatic gain control module. The confidence metric may be based on a percentage of valid audio samples in a given period of time. Based on the confidence metric, the AGC response may be reduced, delayed, frozen, or otherwise altered from the baseline gain adjustment. Time-averaging process may be used to estimate the input signal power level and determine an appropriate baseline gain adjustment. Additionally, weighting functions can be adjusted to prevent overestimation of the signal power.