A method, system and computer software product for managing jitter buffering that accurately measures network latency, the variation in latency (also known as jitter), and efficiently manages the media packet stream and jitter buffers is disclosed. A framer time-stamps incoming packets. A traffic analyzer maintains a sliding window of statistics generated from a recent set of packets. A jitter manager monitors packets, receives and makes adjustments based on information received from the traffic analyzer, and manages any connected jitter buffers.