An audio signal processing apparatus and an audio signal processing method are provide. The audio signal processing apparatus comprises: a plurality of individual audio interfaces, an audio signal processing unit, and an audio channel splitting unit. The audio signal processing unit is utilized for determining a total number of audio channels corresponding to the individual audio interfaces and generating a first output audio signal with a first number of audio channels according to an input audio signal and the total number of audio channels when the audio signal processing apparatus is operated under a first operational mode. The audio channel splitting unit is coupled to the audio signal processing unit and the audio interfaces. When the audio signal processing apparatus is operated under the first operational mode, the audio channel splitting unit splits the first output audio signal with the first number of audio channels to the audio interfaces, respectively.