A generic transaction generator framework for testing a network-based production service may work in conjunction with a product-specific transaction creator module that executes transactions on the service. The transaction creator module may include runtime-discoverable information, such as source code annotations, to communicate product specific details to the framework. Runtime-discoverable information may identify transaction types, transaction methods, data provider methods and data sources. The framework may generate and execute various test transactions and may call a data provider method to prepare data for the transaction and pass the prepared data to a transaction method. The framework may also load and parse test data from a data source and provide the test data to the data provider method for use when preparing data for the transaction.