A system for organizing digital videos to archive and access them at different levels of abstraction uses data available from a closed-caption text along with off-the-shelf natural language processing tools to segment the video into self-contained story sections and speaker blocks. If the subject changes are marked, the system uses these points to divide the video into distinct stories which are represented as nodes attached to the root node in a tree structure and groups speaker segments belonging to a story under the story node as its children.