A human interaction replacement evaluation system analyzes actions taken by a user with an application on a client device that provides features to replace human interaction services with computer-based services. The results of the action provide an indication of the success of a particular action supported by the application (e.g., whether the action has a positive or negative effect on a key performance indicator) or an indication of how likely the user is to be ready to adopt a particular computer-based service. Recommendations are then provided to the user of the application or a manager of the application indicating actions to use, actions that have negative or positive effects on a key performance indicator, and so forth.