公开(公告)号：US20240036530A1

公开(公告)日：2024-02-01

申请号：US17970697

申请日：2022-10-21

申请人： VMWARE, INC.

发明人： MARIUS VILCU ,  SHASHI KUMAR ,  MADAN SINGHAL ,  NICHOLAS MARK GRANT STEPHEN ,  Jad EI-Zein

IPC分类号： G05B13/02

CPC分类号： G05B13/027

摘要： The current document is directed to reinforcement-learning-based controllers and managers that control distributed applications and the infrastructure environments in which they run. The reinforcement-learning-based controllers and managers are both referred to as “management-system agents” in this document. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems. The management-system agents deployed to live, target distributed computer systems operate in a controller mode, in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use. Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for updating and learning optimized policies and value functions, which are then transferred to the deployed management-system agent.