The present invention discloses an inter prediction mode-based image processing method and an apparatus therefor. Specifically, a method for processing an image based on an inter prediction includes: deriving reverse motion information of a current block by using motion information of a block within a reference picture of a current picture; adding the reverse motion information to a motion information candidate list of the current block; deriving motion information of the current block from the motion information selected among motion information candidates added to the motion information candidate list; and generating a prediction block of the current block by using the motion information of the current block, in which the block within the reference picture may be specified by the reverse motion information.