A data validation service may validate data sets maintained for one or more data sources. Several rule sets may describe various rules used to validate one or more data sets. The rule sets may be automatically applied to respective data sets in order to validate the respective data sets according to a dynamically determined schedule for the application of the rule sets. Reporting events may be detected which correspond to a rule set. In response to detecting a reporting event, a responsive action may be performed as described in the rule set, such as providing notification of the reporting event.