Trajectory planning is described for video stabilization. In one example, a sequence of video frames is received and low-pass filtered. If a subset of the low-pass filtered frames are determined to be zero-motion frames, then a smooth trajectory is estimated for the for the zero-motion frames. Jitter is removed from the zero-motion frames using the smooth trajectory and frame warping is applied to the zero-motion frames with jitter removed.