In a packet based multicast video streaming network, a method for minimizing delay in displaying video on a TV set in response to a viewer changing the channel is disclosed. For each live multicast video stream, one or more reference frames and corresponding compressed frames are temporarily stored in a memory in anticipation of the viewer switching to that particular program. When the viewer changes channels, the previously stored reference and compressed frames are immediately read from the memory and rendered for display on the TV. This eliminates the need to wait for the next available reference frame. In other words, this eliminates a temporary blank TV screen each time the channel is changed. Certain compressed frames are dropped and blank frames can be inserted in order to catch up to and synchronize with the live video broadcast. The particular frames to be dropped and the number of blank frames to be inserted is a function of the bandwidth limitation imposed by the last mile of the network.