A system and method for effectuating fast channel change in an adaptive streaming environment. A plurality of adjacent channels may be predicted relative to a current streaming channel engaged by a client device. Upon prioritizing the adjacent channels, the client device may be configured to pre-fetch metadata and initialization information for a subset of the prioritized adjacent channels based on bandwidth conditions, video player buffer status, etc. When a channel is changed, the client device is operative to use the pre-fetched data to quickly obtain encoded media content at a select bitrate for rendering.