A method and apparatus is provided for automatically classifying a multimedia artifact based on scoring, and selecting the appropriate set of ontologies from among all possible sets of ontologies, preferably using a recursive routing selection technique. The semantic tagging of the multimedia artifact is enhanced by applying only classifiers from the selected ontology, for use in classifying the multimedia artifact, wherein the classifiers are selected based on the context of the multimedia artifact. One embodiment of the invention, directed to a method for classifying a multimedia artifact, uses a specified criteria to select one or more ontologies, wherein the specified criteria indicates the comparative similarity between specified characteristics of the multimedia artifact and each ontology. The method further comprises scoring and selecting one or more classifiers from a plurality of classifiers that respectively correspond to semantic element of the selected ontologies, and evaluating the multimedia artifact using the selected classifiers to determine a classification for the multimedia artifact.