Disclosed is a video signal processing method and device for encoding or decoding a video signal. More specifically, disclosed are a video signal processing method and a video signal processing device for performing the method, the method comprising: obtaining reference line information indicating a reference line for intra prediction of a current block among a plurality of reference lines comprising neighboring samples of the current block; determining, on the basis of the reference line information, an intra prediction mode for the current block among a plurality of intra prediction modes constituting an intra prediction mode set; and decoding the current block on the basis of a plurality of reference samples on the reference line according to the reference line information and the determined intra prediction mode.