Audio signal processing apparatus is disclosed. The audio signal processing apparatus includes a first audio signal extracting section, a second audio signal extracting section, a sense-of-depth controlling section, a sense-of-sound-expansion controlling section, a control signal generating section, and a mixing section. The first audio signal extracting section extracts a main audio signal. The second audio signal extracting section extracts a sub audio signal. The sense-of-depth controlling section processes the extracted main audio signal to control a sense of depth. The sense-of-sound-expansion controlling section processes the extracted sub audio signal to vary a sense of sound expansion. The control signal generating section generates a first control signal with which the sense-of-depth controlling section is controlled and a second control signal with which the sense-of-sound-expansion controlling section is controlled. The mixing section mixes an output audio signal of the sense-of-depth controlling section and an output audio signal of the sense-of-sound-expansion controlling section.