A cloud-based service monitoring device includes a criteria database and an exceptions database. The criteria database includes predefined configuration criteria corresponding to approved operating parameters of each cloud-based service being monitored. The exceptions database includes predefined configuration exceptions such that, for a given instance, each configuration exception corresponds to a different instance-specific criteria than the associated configuration criteria for the cloud-based service. The monitoring device extracts configuration settings from instances of the cloud-based service and compares the settings to the configuration criteria of the cloud-based service. If a suspect setting is identified that does not satisfy the configuration criteria at the service level, the monitoring device compares the suspect setting to instance-specific criteria. If the setting does not satisfy both the configuration criteria and the instance-specific criteria, an alert message may be transmitted to an administrator's device.