Obtain, from an existing machine learning classifier, original probabilistic scores classifying samples taken from two or more groups into two or more classes via supervised machine learning. Associate the original probabilistic scores with a plurality of original Lagrange multipliers. Adjust values of the plurality of original Lagrange multipliers via low-dimensional convex optimization to obtain updated Lagrange multipliers that satisfy fairness constraints as compared to the original Lagrange multipliers. Based on the updated Lagrange multipliers, closed-form transform the original probabilistic scores into transformed probabilistic scores that satisfy the fairness constraints while minimizing loss in utility. The fairness constraints are with respect to the two or more groups.