Techniques are described for in-service configuration data migration for distributed micro service-based applications. In one example, a network device comprises a plurality of legacy data repositories comprising configuration data in key-value pair format that specifies a plurality of parameters and corresponding values for operation of the network device, and a hierarchical configuration data model having a plurality of nodes arranged in a hierarchical organization having a root node and a plurality of leaf nodes. Each of the nodes of the configuration data model is configured to store a set of configuration data parameters for the network device. One or more of the nodes includes a plurality of external references to respective parameters of the plurality of parameters stored within the plurality of legacy data repositories. Process circuitry is configured to perform a migration of the configuration data from the legacy data repositories to the hierarchical data model.