In one embodiment, a network assurance service maintains a data lake of network telemetry data obtained by the service from any number of computer networks. The service generates a machine learning model for on-premise execution in a particular computer network to detect network issues in the particular network. To do so, the service repeatedly selects a candidate set of model settings based in part on the data lake of network telemetry data, trains a machine learning model using network telemetry data from the data lake that matches the candidate set of model settings, and tests performance of the trained model using an emulator that emulates network issues in the particular network. The service further deploys the generated machine learning model to the particular computer network for on-premise execution.