A simulation management service receives a request to perform reinforcement learning for a robotic device. The request can include computer-executable code defining a reinforcement function for training a reinforcement learning model for the robotic device. In response to the request, the simulation management service generates a simulation environment and injects the computer-executable code into a simulation application for the robotic device. Using the simulation application and the computer-executable code, the simulation management service performs the reinforcement learning within the simulation environment.