Dialog system training techniques using a simulated user system are described. In one example, a simulated user system supports multiple agents. The dialog system, for instance, may be configured for use with an application (e.g., digital image editing application). The simulated user system may therefore simulate user actions involving both the application and the dialog system which may be used to train the dialog system. Additionally, the simulated user system is not limited to simulation of user interactions by a single input mode (e.g., natural language inputs), but also supports multimodal inputs. Further, the simulated user system may also support use of multiple goals within a single dialog session