Video entity templates defining common features that relate to various metadata types shared among a group of video Web pages are generated for target Web sites. Metadata associated with videos contained within Web pages belonging to a particular target Web site can then be automatically and accurately extracted using a video entity template generated for the particular target Web site. This metadata can then be indexed for use by video search applications in providing video search results.