Techniques are provided for evaluating anomaly detection algorithms using impersonation data derived from user transaction data. An exemplary method comprises obtaining transaction data of a given enterprise organization comprising transactions of a plurality of users; generating impersonation data by modifying one or more features of a subset of the transaction data of the given enterprise organization; classifying (i) at least a portion of the transaction data of the given enterprise organization, and (ii) at least a portion of the impersonation data using the anomaly detection algorithm of the given enterprise organization, wherein records of the impersonation data comprise a known classification; and evaluating a performance of the anomaly detection algorithm of the given enterprise organization by comparing the classification of records of the impersonation data by the anomaly detection algorithm with the known classification.