A semantic-based searching apparatus and method which can search for desired information from various types of media using associative properties of search target objects are provided. The semantic-based searching apparatus stores an associative search structure that previously stores a semantic index. The associative search structure is to obtain a final search object corresponding to an input search query. The semantic index configuration includes feature metadata used to identify a specific object, semantic entity metadata to indicate semantic entities corresponding to the feature metadata, and semantic relation metadata to indicate a relation between the semantic entities. The semantic-based searching apparatus uses semantic metadata stored in conformity with the semantic index configuration, the semantic index, and the associative search structure to generate a search result corresponding to the input search query.