A system for controlling a CVVD by adjusting an actuator for controlling the CVVD is provided. The system includes an electronic control unit (ECU) configured to output a command phase angle and a selection signal for controlling the actuator based on a vehicle state and a mode of operation, and an actuator controller configured to operate based on the command phase angle and the selection signal and output an electric current to adjust the actuator. The actuator controller is configured to output an electric current corresponding to the command phase angle originating from the ECU or adjust the command phase angle and then output an electric current corresponding to the corrected command phase angle based on a type of selection signal.