A playback speed effect is applied to a video using a playback speed template. The playback speed template specifies playback speed ratios (i.e., ratios between playback duration and capture duration) at the highlight moment, at a template start time, and at a template end time. A video associated with a highlight tag indicating a highlight capture time of a highlight moment within the video is accessed. An input portion of the video including the highlight moment is identified. The duration of the input portion has a duration depending on the template start time and the template end time. A playback speed template is applied to the input portion. A modified video including a modified video portion is generated from the input portion of the video according to the applied playback speed template and is provided for subsequent playback.