A software transactional memory system implements a lightweight key-based action framework. The framework includes a set of unified application programming interfaces (APIs) exposed by an STM library that allow clients to implement actions that can be registered, queried, and updated using specific keys by transactions or transaction nests in STM code. Each action includes a key, state information, and a set of one or more callbacks that can be hooked to the validation, commit, abort, and/or re-execution phases of transaction execution. The actions extend the built-in concurrency controls of the STM system with customized control logics, support transaction nesting semantics, and enable integration with garbage collection systems.