- Patent Title: Dynamic hardware selection for experts in mixture-of-experts model
-
Application No.: US15849633Application Date: 2017-12-20
-
Publication No.: US11893502B2Publication Date: 2024-02-06
- Inventor: Nicholas Malaya , Nuwan Jayasena
- Applicant: Advanced Micro Devices, Inc.
- Applicant Address: US CA Santa Clara
- Assignee: Advanced Micro Devices, Inc.
- Current Assignee: Advanced Micro Devices, Inc.
- Current Assignee Address: US CA Santa Clara
- Agency: Volpe Koenig
- Main IPC: G06N5/022
- IPC: G06N5/022 ; G06N20/00 ; G06F7/02

Abstract:
A system assigns experts of a mixture-of-experts artificial intelligence model to processing devices in an automated manner. The system includes an orchestrator component that maintains priority data that stores, for each of a set of experts, and for each of a set of execution parameters, ranking information that ranks different processing devices for the particular execution parameter. In one example, for the execution parameter of execution speed, and for a first expert, the priority data indicates that a central processing unit (“CPU”) executes the first expert faster than a graphics processing unit (“GPU”). In this example, for the execution parameter of power consumption, and for the first expert, the priority data indicates that a GPU uses less power than a CPU. The priority data stores such information for one or more processing devices, one or more experts, and one or more execution characteristics.
Public/Granted literature
- US20190188577A1 DYNAMIC HARDWARE SELECTION FOR EXPERTS IN MIXTURE-OF-EXPERTS MODEL Public/Granted day:2019-06-20
Information query