The invention mainly relates to a design method of a reinforcement learning based optimal tracking controller for a model unknown servo system. The design method of the reinforcement learning based optimal tracking controller for the model unknown servo system is introduced mainly on the basis of a simplified reinforcement learning evaluation-execution structure with a high-order neural network approach method, and the optimal tracking control solution speed of a motor is increased. As for the model unknown servo system, firstly, homeostatic control is solved with a multilayer neutral networkintelligent identification system model; performance indexes are given, and a high-order neutral network approach optimal performance index function is applied; an HJB (Hamilton-Jacobi-Bellman) equation is established according to an approximate performance index function and the identification system model, and the optimal feedback control of the servo system is solved. The optimal tracking control is calculated according to the solved homeostatic control and optimal feedback control, so that tracking error accumulation values and system energy consumption are minimized simultaneously while load rotation angle and rotation speed rapidly track given signals.