公开(公告)号：US20240046069A1

公开(公告)日：2024-02-08

申请号：US17970830

申请日：2022-10-21

Applicant: VMWARE, INC.

Inventor： MARIUS VILCU ,  Peter Rudy ,  Asmitha Rathis ,  Aiswaryaa Venugopalan

IPC: G06N3/04

CPC classification number: G06N3/0454

Abstract: The current document is directed to reinforcement-learning-based management-system agents that control distributed applications and the infrastructure environments in which they run. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems where they operate in a controller mode in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use. Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for optimizing its policy and value functions. To further ensure safe operational control of the environment, the management-system agents employ lookahead planning, action budgets, and action constraints to forestall issuance, by management-system controllers, of potentially deleterious actions.

公开(公告)号：US20240037495A1

公开(公告)日：2024-02-01

申请号：US18097522

申请日：2023-01-17

Applicant: VMWARE, INC.

Inventor： NICHOLAS MARK GRANT STEPHEN ,  MARIUS VILCU ,  PRAHALAD DESHPANDE ,  SANTOSHKUMAR KAVADIMATTI

IPC: G06Q10/087 ,  G06Q10/04

CPC classification number: G06Q10/087 ,  G06Q10/04

Abstract: The current document is directed to a meta-level management system (“MMS”) that aggregates information and functionalities provided by multiple management systems and provides additional management functionalities and information. In one implementation, the MMS interfaces to external entities and users through an MMS application programming interface (“API”) implemented as a GraphQL™ interface. The MMS API, in turn, accesses microservices and stream/batch processing components through microservice and stream/batch-processing-component GraphQL interfaces. The MMS employs at least three different databases: (1) an inventory/configuration database; (2) a metrics database that stores metrics derived from time-series data obtained from the multiple management systems and from other information stored in the inventory/configuration database; and (3) an MMS database that stores business insights and other MMS-generated data. A central data bus is implemented by a KAFKA™ event-streaming system. The data and information is input to the data bus by the various microservices, stream/batch processing components, and collectors.

公开(公告)号：US20240036530A1

公开(公告)日：2024-02-01

申请号：US17970697

申请日：2022-10-21

Applicant: VMWARE, INC.

Inventor： MARIUS VILCU ,  SHASHI KUMAR ,  MADAN SINGHAL ,  NICHOLAS MARK GRANT STEPHEN ,  Jad EI-Zein

IPC: G05B13/02

CPC classification number: G05B13/027

Abstract: The current document is directed to reinforcement-learning-based controllers and managers that control distributed applications and the infrastructure environments in which they run. The reinforcement-learning-based controllers and managers are both referred to as “management-system agents” in this document. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems. The management-system agents deployed to live, target distributed computer systems operate in a controller mode, in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use. Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for updating and learning optimized policies and value functions, which are then transferred to the deployed management-system agent.