The apparatus of the invention calculates filter coefficients for controlling sound field and sound image, based on a plurality of first impulse response signals and a pair of second impulse response signals. The plurality of first impulse response signals indicate impulse responses from loudspeakers reproducing audio signals to both ears of a listener. The pair of second impulse response signals indicate impulse responses from a reference loudspeaker at a position at which a sound image is localized to both ears of the listener. The apparatus includes: a feature extracting section for receiving the pair of second impulse response signals, for extracting parameters representing features of the pair of second impulse response signals, and for outputting parameter signals; a signal adjusting section for adjusting at least one of the plurality of first impulse response signals based on the parameter signals, and for outputting a pair of third impulse response signals having the same features as the extracted features; and a coefficient calculating section for calculating the filter coefficients for controlling the sound field and sound image, based on the plurality of first impulse response signals and the pair of third impulse response signals applied from the signal adjusting means.