Technologies are shown for application risk assessment in an authentication service where an authorization request is received from a third party application calling an Application Programming Interface (API). Risk assessment policies that pertain to behavioral characteristics, such as API usage patterns or past delegation of permissions, are applied to the authorization request to obtain a risk assessment score. If the risk assessment score does not exceed a risk threshold, then an authorization message is sent in response to the authorization request. If the risk assessment score exceeds the risk threshold, then remedial action, such as suspending the application, limiting the available actions, or sending a notification to a trusted security application, is executed for an account associated with the third party application. Machine learning can be applied to historical behavioral data to generate the risk assessment policies.