A model parallel training technique for neural architecture search including the following operations: (i) receiving a plurality of ML (machine learning) models that can be substantially interchangeably applied to a computing task; (ii) for each given ML model of the plurality of ML models: (a) determining how the given ML model should be split for model parallel processing operations, and (b) computing a model parallelism score (MPS) for the given ML model, with the MPS being based on an assumption that the split for the given ML model will be used at runtime; and (iii) selecting a selected ML model based, at least in part, on the MPS scores of the ML models of the plurality of ML models.