A web crawler indexes documents including information about document contents and metadata including information such as a URL. However, some applications rely on URL's that change frequently or are constructed to include user information so that the contents retrieved is customized to the user. An approach is provided for storing generic URL's in an index at crawl time, which are customized for the user at search time. A callback mechanism may be used to dynamically transform the generic URL into a URL that is specific to the user issuing the query and/or includes current information that may change frequently. In this way, when the query or search results are returned to the user, the user receives links that are active and valid for that particular user, directing the user to the appropriate site, application, etc. without requiring continuous updating of a very large index.