A system and a method for self calibrating a voltage-controlled oscillator (VCO). In the system, a mode controller generates a control signal for each of an automatic band selection mode, an automatic gain tuning mode, and a phase-locking mode, from a frequency comparison result between a reference clock signal and a divided clock signal which is generated by dividing a frequency of an oscillation signal, and thereby controls the VCO, so that the VCO may generate the oscillation signal which is automatically phase-locked in a target frequency with an optimal state.