The invention provides a system and method for controlling use of handset communication device for communication by driver. A system in accordance with an embodiment includes: an obtaining component for obtaining driving state information of the vehicle; a directional antenna for capturing a wireless communication signal of the handset communication device around the driver; a judging component for judging whether a driving state of the vehicle is in an unstable state based on predefined criteria and the driving state information of the vehicle; and a control component for transmitting a remote control signal, based on a judging result of the judging component, to intervene in a communication of the handset communication device during a call.