In the present disclosure, a method of decoding a video signal and a device therefor are disclosed. Specifically, a method of decoding an image based on an inter prediction mode includes deriving a motion vector of an available spatial neighboring block around a current block; deriving a collocated block of the current block based on the motion vector of the spatial neighboring block; deriving a motion vector in a sub-block unit in the current block based on a motion vector of the collocated block; and generating a prediction block of the current block using the motion vector derived in the sub-block unit, wherein the collocated block may be specified by the motion vector of the spatial neighboring block in one pre-defined reference picture.