Systems, methods, and computer-readable for multi-temporal scale analysis include obtaining two or more timescales associated with one or more images. A context associated with a monitoring objective is obtained, based on real time analytics or domain specific knowledge. The monitoring objective can include object detection, event detection, pattern recognition, or other. At least a subset of timescales for performing a differential analysis on the one or more images is determined based on the context. Multi timescale surprise detection and clustering are performed using the subset of timescales to determine whether any alerts are to be generated based on entropy based surprises. A set of rules can be created for the monitoring objective based on the differential analytics and alerts or entropy based surprises, if any.