An audio signal processing apparatus and method for processing an audio signal are provided. The audio signal processing apparatus includes a signal detector configured to detect a long-term envelope and a short-term envelope of a lower range signal of an input signal, and a controller configured to control a gain in a beat section of the input signal, wherein the beat section is determined based on the detected long-term envelope and the short-term envelope of the lower range signal.