An audio signal processing apparatus includes: a division section that divides at least two or more channel audio signals into components in a plurality of frequency bands; a phase difference calculation section that calculates a phase difference between the two or more channel audio signals at each the frequency band; a level ratio calculation section that calculates a level ratio between the two or more channel audio signals at each the frequency band; a sound image localization estimation section that estimates, based on the level ratio or the phase difference, sound image localization at each the frequency band; and a control section that controls the estimated sound image localization at each the frequency band by adjusting the level ratio or the phase difference.