The present invention is a method and system for generation and playback of supplemented videos which include interactive features. The supplemented video includes hotspots that allow a video viewer to interact with the video when the video viewer sees an object. The hotspots can be manually defined. Information regarding the object and the hotspot can be stored in a separate XML file. Furthermore, the present invention can be a marketplace where a desired object can be found by searching the XML file. The search results can list the supplemented videos which contain hotspots corresponding to the object and also the time in the supplemented video in which the object is found. The present invention can also aggregate data about the objects based on the playback of the supplemented videos and the video viewer's interaction with the supplemented videos.