A computer-implemented method of training a machine learning model to prevent data leakage from membership inference attacks. A pre-trained model and a pre-defined hyperparameter λ are received as an input. A forward pass is applied by querying the pre-trained model with a private data. An initial loss distribution LINIT of loss values is computed. A batch loss of a minibatch from the private data is computed after beginning a fine-tuning operation to transform the pre-trained model into a fine-tuned model, and a batch loss distribution LBATCH is computed. A divergence metric is computed between LINIT and LBATCH, and the output of the divergence metric is multiplied with the pre-defined hyperparameter A to obtain a result that is added to the batch loss as a regularizer. The model parameters are updated by computing backpropagation on the regularized loss. The fine-tuned model is output.