A method for encoding a current frame of a video includes jointly determining respective motion models for reference frames and encoding the current frame using the respective motion models. The reference frames are used for encoding the current frame. Jointly determining respective motion models for reference frames includes determining respective aggregated residuals for combinations of candidate motion models and selecting the combination of candidate motion models that corresponds to the smallest aggregated residual. The respective motion models correspond to the candidate motion models of the selected combination.