Methods, computer readable media, devices, and systems provide for distributed machine learning. In one aspect, a method of training a model is disclosed. The method includes receiving, by a client device, from one or more servers, an intermediate model, training, by the client device, the intermediate model based on private data, and transmitting, by the client device, to the one or more servers, the trained intermediate model.