A method for increasing the resolution of a series of low resolution frames of a low resolution video sequence to a series of high resolution frames of a high resolution video sequence includes receiving the series of low resolution frames of the video sequence. The system determines a first plurality of semantically relevant key points of a first low resolution frame of the series of low resolution frames and determines a second plurality of semantically relevant key points of a second low resolution frame of the series of low resolution frames. The system temporally processes the first plurality of key points based upon the second plurality of key points to determine a more temporally consistent set of key points for the first plurality of key points.