An input picture signal having noise added through a transmission path or the like is supplied. Noise is removed from the input picture by a class categorizing adaptive process in which predictive coefficients are pre-learnt and decided for each class. A class corresponding to a noise component contained in the input picture is decided. Predictive coefficients of the class and the values of pixels of frames containing a considered pixel of the input picture signal are linearly combined. Thus, predictive pixel values are generated, which are free of noise. When motion of a considered pixel is detected and pixels that are used to decide a class and pixels that are used for a predictive calculation are compensated corresponding to the motion, noise accurately corresponding to a noise component can be removed from the input picture signal.