A computer-implemented method, according to one approach, includes issuing a hyperparameter optimization (HPO) query to a plurality of computing devices. HPO results are received from the plurality of computing devices, and the HPO results include a set of hyperparameter (HP)/rank value pairs. The method further includes computing, based on the set of HP/rank value pairs, a global set of HPs from the HPO results for federated learning (FL) training. An indication of the global set of HPs is output to the plurality of computing devices. A computer program product, according to another approach, includes a computer readable storage medium having program instructions embodied therewith. The program instructions are readable and/or executable by a computer to cause the computer to perform the foregoing method.