An interactive video system is disclosed that processes a video data stream and an associated data stream corresponding to the video data stream. The interactive video system displays a video image defined by the video data stream on a display device and performs interactive command functions specified by the associated data stream. The interactive command functions include commands that specify placement of a video display window, commands that specify parameters of graphical objects that are associated with the video image and commands that specify pixel data or graphics description for the graphical object and commands for placement of selection windows and that specify interactive functions for the selection windows.