An audio jack connector adapted for receiving an audio plug connector with a tip end includes an insulating housing and a contact terminal. The insulating housing defines a plug insertion hole from a front end to a rear end thereof and a recess communicating with the plug insertion hole. The contact terminal has a connecting portion received in a rear of the recess. Two opposite ends of the connecting portion are extended downward to form first and second holding arms. Portions of the holding arms extend along the axis of the plug insertion hole to form first and second locating boards. Edges of the locating boards extend towards each other and are bent along the axis of the plug insertion hole to form first and second elastic arms. First and second contact portions are protruded into the plug insertion hole from free ends of the elastic arms.