The present technology relates to an acoustic signal processing device, an acoustic signal processing method and a program with which it is possible to expand the variation of configuration of a virtual surround system that stabilizes a sense of localization of a virtual speaker. A crosstalk correction process is performed on a first binaural signal based on a sound source counter side HRTF and asecond binaural signal based on a sound source side HRTF, a first acoustic signal and a second acoustic signal are generated, and the components of a first band in which a first notch of the sound source counter side HRTE appears and a second band in which a second notch appears are attenuated in an input signal or the second binaural signal, the components of the first band and second band of thefirst binaural signal and the second binaural signal being attenuated thereby. An auxiliary signal comprising the component of a third band of the input signal or the second binaural signal having had the components of the first band and second band attenuated, is added to the first acoustic signal, and a third acoustic signal is generated. The present technology can be applied, for example, to an AV amplifier.