An apparatus and method for detecting potentially-improper call behavior (e.g., SPIT, etc.) are disclosed. The illustrative embodiment of the present invention is based on finite-state machines (FSMs) that represent the legal states and state transitions of communications protocols at nodes during Voice over Internet Protocol (VoIP) calls. In accordance with the illustrative embodiment, a library of FSM execution profiles associated with improper call behavior and a set of rules (or rule base) associated with improper FSM behavior over one or more calls are maintained. When the behavior of one or more finite-state machines during one or more calls matches either an execution profile in the library or a rule in the rule base, an alert is generated.