METHODS AND SYSTEMS THAT USE MACHINE-LEARNING TO DETERMINE WORKLOADS AND TO EVALUATE DEPLOYMENT/CONFIGURATION POLICIES

    公开(公告)号:US20230177345A1

    公开(公告)日:2023-06-08

    申请号:US17545403

    申请日:2021-12-08

    Applicant: VMware, Inc.

    CPC classification number: G06N3/088 G06N3/04 G06F9/505

    Abstract: The current document is directed to methods and systems that determine workload characteristics of computational entities from stored data and that evaluate deployment/configuration policies in order to facilitate deploying, launching, and controlling distributed applications, distributed-application components, and other computational entities within distributed computer systems. Deployment/configuration policies are powerful tools for assisting managers and administrators of distributed applications and distributed computer systems, but constructing deployment/configuration policies and, in particular, evaluating the relative effectiveness of deployment/configuration policies in increasingly complex distributed-computer-system environments may be difficult or practically infeasible for many administrators and managers and may be associated with undesirable or intolerable levels of risk. The currently disclosed machine-learning-based deployment/configuration-policy evaluation methods and systems represent a significant improvement to policy-based management and control that address both of these problems.

    Administrator-monitored reinforcement-learning-based application manager

    公开(公告)号:US10922092B2

    公开(公告)日:2021-02-16

    申请号:US16518617

    申请日:2019-07-22

    Applicant: VMware, Inc.

    Abstract: The current document is directed to an administrator-monitored reinforcement-learning-based application manager that can be deployed in various different computational environments to manage the computational environments with respect to one or more reward-specified goals. Certain control actions undertaken by the administrator-monitored reinforcement-learning-based application manager are first proposed, to one or more administrators or other users, who can accept or reject the proposed control actions prior to their execution. The reinforcement-learning-based application manager can therefore continue to explore the state/action space, but the exploration can be parametrically constrained as well as by human-administrator oversight and intervention.

    Safe-operation-constrained reinforcement-learning-based application manager

    公开(公告)号:US11042640B2

    公开(公告)日:2021-06-22

    申请号:US16502587

    申请日:2019-07-03

    Applicant: VMware, Inc.

    Abstract: The current document is directed to a safe-operation-constrained reinforcement-learning-based application manager that can be deployed in various different computational environments, without extensive manual modification and interface development, to manage the computational environments with respect to one or more reward-specified goals. Control actions undertaken by the safe-operation-constrained reinforcement-learning-based application manager are constrained, by stored action filters, to constrain state/action-space exploration by the safe-operation-constrained reinforcement-learning-based application manager to safe actions and thus prevent deleterious impact to the managed computational environment.

    Adversarial automated reinforcement-learning-based application-manager training

    公开(公告)号:US10977579B2

    公开(公告)日:2021-04-13

    申请号:US16518807

    申请日:2019-07-22

    Applicant: VMware, Inc.

    Abstract: The current document is directed to automated reinforcement-learning-based application managers that that are trained using adversarial training. During adversarial training, potentially disadvantageous next actions are selected for issuance by an automated reinforcement-learning-based application manager at a lower frequency than selection of next actions, according to a policy that is learned to provide optimal or near-optimal control over a computing environment that includes one or more applications controlled by the automated reinforcement-learning-based application manager. By selecting disadvantageous actions, the automated reinforcement-learning-based application manager is forced to explore a much larger subset of the system-state space during training, so that, upon completion of training, the automated reinforcement-learning-based application manager has learned a more robust and complete optimal or near-optimal control policy than had the automated reinforcement-learning-based application manager been trained by simulators or using management actions and computing-environment responses recorded during previous controlled operation of a computing-environment.

    Computationally efficient reinforcement-learning-based application manager

    公开(公告)号:US10949263B2

    公开(公告)日:2021-03-16

    申请号:US16518717

    申请日:2019-07-22

    Applicant: VMware, Inc.

    Abstract: The current document is directed to automated reinforcement-learning-based application managers that obtain increased computational efficiency by reusing learned models and by using human-management experience to truncate state and observation vectors. Learned models of managed environments that receive component-associated inputs can be partially or completely reused for similar environments. Human managers and administrators generally use only a subset of the available metrics in managing an application, and that subset can be used as an initial subset of metrics for learning an optimal or near-optimal control policy by an automated reinforcement-learning-based application manager.

    Modular reinforcement-learning-based application manager

    公开(公告)号:US10802864B2

    公开(公告)日:2020-10-13

    申请号:US16261253

    申请日:2019-01-29

    Applicant: VMware, Inc.

    Abstract: The current document is directed to a modular reinforcement-learning-based application manager that can be deployed in various different computational environments without extensive modification and interface development. The currently disclosed modular reinforcement-learning-based application manager interfaces to observation and action adapters and metadata that provide a uniform and, in certain implementations, self-describing external interface to the various different computational environments which the modular reinforcement-learning-based application manager may be operated to control. In addition, certain implementations of the currently disclosed modular reinforcement-learning-based application manager interface to a user-specifiable reward-generation interface to allow the rewards that provide feedback from the computational environment to the modular reinforcement-learning-based application manager to be tailored to meet a variety of different user expectations and desired control policies.

    MULTI-DIMENSIONAL AUTO SCALING OF CONTAINER-BASED CLUSTERS

    公开(公告)号:US20240419457A1

    公开(公告)日:2024-12-19

    申请号:US18210032

    申请日:2023-06-14

    Applicant: VMware, Inc.

    Abstract: The disclosure provides a method for determining a target configuration for a container-based cluster. The method generally includes determining, by a virtualization management platform configured to manage components of the cluster, a current state of the cluster, determining, by the virtualization management platform, at least one of performance metrics or resource utilization metrics for the cluster based on the current state of the cluster, processing, with a model configured to generate candidate configurations recommended for the cluster, the current state and at least one of the performance metrics or the resource utilization metrics and thereby generate the candidate configurations, calculating a reward score for each of the candidate configurations, selecting the target configuration as a candidate configuration from the candidate configurations based on the reward score of the target configuration, and adjusting configuration settings for the cluster based on the target configuration to alter the current state of the cluster.

    REINFORCEMENT-LEARNING-BASED DISTRIBUTED-APPLICATION CONTROLLER INCORPORATING TRANSFER LEARNING

    公开(公告)号:US20230161635A1

    公开(公告)日:2023-05-25

    申请号:US17532876

    申请日:2021-11-22

    Applicant: VMware, Inc.

    CPC classification number: G06F9/5055 G06K9/6262 G06N3/0454

    Abstract: The current document is directed to a reinforcement-learning-based application manager that controls the operation of one or more applications and that employs transfer learning to improve initialization and operation of the reinforcement-learning-based application manager and to improve operation of the one or more distributed computer systems that host the applications controlled by the reinforcement-learning-based application manager. Transfer learning, in the disclosed implementations, is achieved by logically decomposing machine-learning-based function approximators for reinforcement-learning functions into component-specific function approximators, storing pre-trained function approximators and pre-trained component-specific function approximators, and initializing function approximators for reinforcement-learning-based application managers using the stored pre-trained function approximators and pre-trained component-specific function approximators.

    SIMULATOR-TRAINING FOR AUTOMATED REINFORCEMENT-LEARNING-BASED APPLICATION-MANAGERS

    公开(公告)号:US20200065704A1

    公开(公告)日:2020-02-27

    申请号:US16518845

    申请日:2019-07-22

    Applicant: VMware, Inc.

    Abstract: The current document is directed to methods and systems for simulation-based training of automated reinforcement-learning-based application managers. Simulators are generated from data collected from controlled computing environments controlled and may employ any of a variety of different machine-learning models to learn state-transition and reward models. The current disclosed methods and systems provide facilities for visualizing aspects of the models learned by a simulator and for initializing simulator models using domain information. In addition, the currently disclosed simulators employ weighted differences computed from simulator-generated and training-data state transitions for feedback to the machine-learning models to address various biases and deficiencies of commonly employed difference metrics in the context of training automated reinforcement-learning-based application managers.

Patent Agency Ranking