A service handover control apparatus and a method using the apparatus are provided. The service handover control apparatus includes a service handover management unit collecting information associated with a transmitting terminal and a wireless access network, which operate based on an Internet Protocol multimedia subsystem (IMS), and outputting a control signal which controls service handovers of the transmitting terminal and the receiving terminal based on the collected information; and a service handover controller providing the service handover management unit with the information associated with the transmitting terminal and the receiving terminal, and controlling the service handovers of the transmitting terminal and the receiving terminal based on the control signal.