A computer implemented method for deriving an attribute entity network (AEN) from video data is disclosed, comprising the steps of extracting at least two entities from the video data, tracking the trajectories of the at least two entities to form at least two tracks, deriving at least one association between at least two entities by detecting at least one event involving the at least two entities, where the detecting of at least one event is based on detecting at least one spatiotemporal motion correlation between the at least two entities, and constructing the AEN by creating a graph wherein the at least two objects form at least two nodes and the at least one association forms a link between the at least two nodes.