In various examples, systems and methods are disclosed that train a machine learning model(s)—such as a large language model (LLM)—for one or more specific domains. In some embodiments, the machine learning model(s) may include at least a base model(s) as well as additional parts, such as additional layers, associated with the domains for which the machine learning model(s) is being trained. As such, the parts of the machine learning model(s) may be trained separately, such that training data associated with a domain is used to train a part of the machine learning model(s) that is associated with the domain without training the other part(s) of the machine learning model(s). The systems and methods may then use these parts when deploying the machine learning model(s), such as by activating and/or deactivating parts based on the input data being processed.