The currently existing ISO/AVC file format is modified by providing extensions to store and access video content currently being defined by the SVC standard. Specifically, extensions to the AVC file format are made to provide a new SVC file format that enables the storage and access of scalable video data. The scalable video data is stored as a single track within a media data section of the SVC file format. New extensions are defined for description entries and boxes within a metadata section of the SVC file format. These extensions provide means for extracting sub-streams or layers from the single track of scalable video data stored in the media data section.