The present invention is a method of representing video data in a non-linear paradigm. Video data are categorized into semantic content comprising multi-layer structure each denotes semantic reference, such as different cinematic entities. The semantic content is organized in a hierarchical structure where the top layer denotes global information while the lowest layer represents primitive information. The cinematic entities in the top layer are hyper-linked to the entities in the second layer. The entities in the second layer are hyper-linked to the third layer and so forth. Each cinematic entity in the lowest layer is designated to a part of the video content and hyper-linked to the corresponding video data. The semantic content comprises hyper-linked video data in an N-to-N relationship. N-to-N relationship means the data are hyper-video and the video data supports multiple access and multiple presentation. An apparatus for presenting categorized semantic content is also disclosed.