An image processing device includes an extraction unit, a judgment unit, a restoration unit and a specification unit. The extraction unit specifies a first same-color area having a color similar to a color of a pointer in a pre-pointed image of a subject not overlapped by the pointer to extract a pointer portion from a being-pointed image of the subject overlapped by the pointer on the basis of the first same-color area. The restoration unit restores a dividing portion of the pointer portion using a surrounding image of the dividing portion when the judgment unit judges that the pointer portion is divided. The specification unit specifies a pointed position pointed with the pointer on the subject on the basis of a shape of the restored pointer portion.