公开(公告)号：US20240036530A1

公开(公告)日：2024-02-01

申请号：US17970697

申请日：2022-10-21

Applicant: VMWARE, INC.

Inventor： MARIUS VILCU ,  SHASHI KUMAR ,  MADAN SINGHAL ,  NICHOLAS MARK GRANT STEPHEN ,  Jad EI-Zein

IPC: G05B13/02

CPC classification number: G05B13/027

Abstract: The current document is directed to reinforcement-learning-based controllers and managers that control distributed applications and the infrastructure environments in which they run. The reinforcement-learning-based controllers and managers are both referred to as “management-system agents” in this document. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems. The management-system agents deployed to live, target distributed computer systems operate in a controller mode, in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use. Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for updating and learning optimized policies and value functions, which are then transferred to the deployed management-system agent.