An audio signal processing apparatus includes first, second and third omni-directional microphones each of which receives sound and generates an omni-directional audio signal and which are spaced apart by a predetermined distance, a first adder section that adds audio signals generated by the first, second and third omni-directional microphones and generates an audio signal having an omni-directivity in the whole circumferential direction, a first subtractor section that subtracts audio signals generated by the first and third omni-directional microphones and generates an audio signal having a directivity in the right-left direction, a second adder section that adds audio signals generated by the first and third omni-directional microphones, a second subtractor section that subtracts an audio signal generated by the second omni-directional microphone from the audio signal added by the second adder section and generates an audio signal having a directivity in the front-back direction, and an output section that adds the audio signal resulting from the multiplication of the audio signal having a directivity in the whole circumferential direction by a predetermined coefficient, the audio signal resulting from the multiplication of the audio signal having a directivity in the right-left direction by a predetermined coefficient, and the audio signal resulting from the multiplication of the audio signal having a directivity in the front-back direction by a predetermined coefficient and generates a unidirectional audio signal.