Testing for bias in a machine learning (ML) model in a manner that is independent of the code/weights deployment path is described. If bias is detected, an alert for bias is generated, and optionally, the ML model can be incrementally re-trained to mitigate the detected bias. Re-training the ML model to mitigate the bias may include enforcing a bias cost function to maintain a level of bias in the ML model below a threshold bias level. One or more statistical metrics representing the level of bias present in the ML model may be determined and compared against one or more threshold values. If one or more metrics exceed corresponding threshold value(s), the level of bias in the ML model may be deemed to exceed a threshold level of bias, and re-training of the ML model to mitigate the bias may be initiated.