In interpolation lines having plural angles passing through an attentional pixel on a scanning line in one field to be interpolated having pixels on an upper scanning line placed over the attentional pixel and pixels on a lower scanning line placed under the attentional pixel, a horizontal spatial frequency is detected in the vicinity of the attentional pixel to be interpolated, interpolation directions are derived by plural interpolation direction decision steps on the basis of obtained spatial frequency information, and the scanning line is interpolated by selecting one direction of the plural interpolation directions.