Disclosed is an ultrasound color Doppler image system. The ultrasound color Doppler image system includes a calculation unit that calculates a mean value associated with an I/Q signal corresponding to a pixel of a color image in an ultrasound image and generates a multiplication value using the calculated mean value; a comparison unit that compares the generated multiplication value with the mean value; and a masking unit that performs masking of the pixel based on a comparison result, and, the calculation unit calculates the mean value of the I/Q signal for each frame, selects a reference mean value based on scales of the calculated mean values, and generates the multiplication value by multiplying the selected reference mean value and a scale factor.