A color optical pen includes a tip unit, a pen body unit attached to the tip unit; a pressure sensor that is disposed in the tip unit and configured to sense at least contact between a display unit of a terminal device and the tip unit; a light source that is disposed in the pen body unit and is configured to output light through the tip unit, if the pressure sensor senses the contact; a color selection switch that is disposed on the pen body, the color selection switch configured to select a color in response to operation by a user; and a driver configured to drive the light source at a frequency or pattern based on operation of the color selection switch.