Media streaming with smooth fast-forward and rewind is disclosed. One embodiment receives a media stream encoded at a first bit rate over a network connection, and in response to an input to fast-forward or rewind the media stream, the media stream may be retrieved at a second bit rate, and rendered at the second bit rate for fast-forward or rewind playback. By receiving the media stream at multiple bit rates, a media stream may be rendered at increased forward or reverse speeds without dropping frames in the media stream.