An object of the disclosure is to provide flexible and highly accurate activity detection means. Provided is an activity detection device including: an input unit that inputs an image sequence including a first image and a second image; an object detection unit that detects a first object in the image sequence; a component model unit that generates first characteristic information characterizing the first object and includes at least one individually trainable component model; and an activity detection unit that generates a first object state corresponding to the first object in the first image and a second object state corresponding to the first object in the second image based on the first characteristic information and determines an activity related to the first object based on the first and second object states.