A hands-free telephone conversation system includes a communication device having a microphone that receives a voice from a user, a speaker that outputs the other party's voice, a control unit that detects the status of the wireless communication, and a local wireless communication unit that wirelessly communicates with a relay device. The relay device includes a microphone that receives the voice from the user, a local wireless communication unit that wirelessly communicates with the communication device, a communication control unit that detects the status of the wireless communication, a microphone switch unit that switches the voice input reception of the microphone, and a network communication unit that sends and receives the voice to and from the other party's communication device through a network.