Embodiments are directed to providing an identity risk score as part of an authentication assertion, applying operating heuristics to determine an operating application's validity and to providing identity risk scores to requesting third parties. In one scenario, an authentication server receives from a cloud service portal various user credentials from a user. The user credentials identify a user to the authentication server. The authentication server verifies the user's identity using the received credentials and generates an identity risk score based on one or more identity factors. The identity factors indicate a likelihood that the user is a valid user. The authentication server encapsulates the generated identity risk score in an authentication assertion and sends the authentication assertion that includes the generated identity risk score to the cloud service portal.