A system and method for contextually enriching a concept database. The method includes determining, based on at least one signature of a first multimedia data element (MMDE) and signatures of a plurality of third concepts stored in the concept database, at least one matching first concept among the plurality of third concepts; generating a reduced representation of the first MMDE; comparing the reduced representation of the first MMDE to signatures representing a plurality of second MMDEs to determine a plurality of matching MMDEs among the plurality of second MMDEs; generating, based on the reduced representation of the first MMDE and the signatures representing the plurality of matching MMDEs, a second concept; and generating at least one context based on the second concept and the plurality of third concepts, wherein each context includes at least one common pattern among the second concept and at least one of the plurality of third concepts.