A method and apparatus for deriving a sub-block motion vector for the current sub-block based on a motion-model function depending on the current sub-block location are disclosed. The derived sub-block motion vector is then used for encoding or decoding the sub-block. The motion-model function may correspond to an affine motion-model function or a bilinear motion-model function. In one embodiment, a new Merge mode can be used to apply prediction of a current block by applying prediction on the sub-block basis using the sub-block motion vector derived from the motion-model function. In another embodiment, an additional inter prediction mode can be used to apply prediction of a current block by applying prediction on the sub-block basis using the sub-block motion vector derived from the motion-model function.