A system and method for generating a concept database based on at least two multimedia data elements (MMDEs). The method includes: generating at least two items from a received MMDE of the at least two MMDEs; determining the items that are of interest for signature generation; generating at least one signature responsive to at least one item of interest of the received MMDE; clustering at least two signatures received from the signature generator responsive of the plurality of MMDEs; reducing the number of signatures in each cluster to a create a signature reduced cluster (SRC) of the cluster; associating metadata with the SRC to a concept structure including at least two SRCs and their associated metadata; and generating at least one index for mapping the received MMDE to at least one concept structure, wherein the concept database includes concept structures and the generated indices for the at least two MMDEs.