This invention relates to an associated video retrieving method and a method therefor which enable a user to freely and easily make associated video retrieval. Video display apparatus having a video display region for visualizing video, an index display region for displaying index information of the videos, point detection unit for detecting a point position for these display regions, object management unit for registering in advance matters or sound in an animation appearing in the video, and control unit for determining a condition of the video to be next playbacked from point information obtained by the point detection unit and from logical structure description of the video separately built up, are provided. Even when a desired scene cannot be specified from the index information, a scene in which any hints relating to the desired scene is found out and associated video retrieval is effected on the basis of such a scene so as to finally obtain the desired scene. Versatile video retrieval can be made by unitedly using each display.