公开(公告)号：US20240037193A1

公开(公告)日：2024-02-01

申请号：US17970726

申请日：2022-10-21

Applicant: VMWARE, INC.

Inventor： Gagandeep SINGH ,  Nina NARODYTSKA ,  Marius VILCU ,  Asmitha RATHIS ,  Arnav CHAKRAVARTHY

IPC: G06F21/16 ,  G06N7/00 ,  G06N3/04

CPC classification number: G06F21/16 ,  G06N7/005 ,  G06N3/0454

Abstract: The current document is directed to reinforcement-learning-based management-system agents that control distributed applications and the infrastructure environments in which they run. Management-system agents are initially trained in simulated environments and specialized training environments before being deployed to live, target distributed computer systems where they operate in a controller mode in which they do not explore the control-state space or attempt to learn better policies and value functions, but instead produce traces that are collected and stored for subsequent use. Each deployed management-system agent is associated with a twin training agent that uses the collected traces produced by the deployed management-system agent for optimizing its policy and value functions. When the optimized policy is determined to be more robust, stable, and effective than the policy of the corresponding deployed management-system agent, the optimized policy is transferred to the deployed management-system agent.

公开(公告)号：US20230161635A1

公开(公告)日：2023-05-25

申请号：US17532876

申请日：2021-11-22

Applicant: VMware, Inc.

Inventor： Marius VILCU ,  Dongni Wang ,  Asmitha Rathis ,  Greg Burk

IPC: G06F9/50 ,  G06K9/62 ,  G06N3/04

CPC classification number: G06F9/5055 ,  G06K9/6262 ,  G06N3/0454

Abstract: The current document is directed to a reinforcement-learning-based application manager that controls the operation of one or more applications and that employs transfer learning to improve initialization and operation of the reinforcement-learning-based application manager and to improve operation of the one or more distributed computer systems that host the applications controlled by the reinforcement-learning-based application manager. Transfer learning, in the disclosed implementations, is achieved by logically decomposing machine-learning-based function approximators for reinforcement-learning functions into component-specific function approximators, storing pre-trained function approximators and pre-trained component-specific function approximators, and initializing function approximators for reinforcement-learning-based application managers using the stored pre-trained function approximators and pre-trained component-specific function approximators.