Techniques and mechanisms are disclosed for configuring actions to be performed by a network security application in response to the detection of potential security incidents, and for causing a network security application to report on the performance of those actions. For example, users may use such a network security application to configure one or more “modular alerts.” As used herein, a modular alert generally represents a component of a network security application which enables users to specify security modular alert actions to be performed in response to the detection of defined triggering conditions, and which further enables tracking information related to the performance of modular alert actions and reporting on the performance of those actions.