中科微点-全球专利检索

  1. Login
  2. Sign Up

Search Results

23 records (took 0.026 seconds)

    ADVERSARIAL AUTOMATED REINFORCEMENT-LEARNING-BASED APPLICATION-MANAGER TRAINING
    11.
    发明申请
    ADVERSARIAL AUTOMATED REINFORCEMENT-LEARNING-BASED APPLICATION-MANAGER TRAINING 审中-公开

    公开(公告)号：US20200065703A1

    公开(公告)日：2020-02-27

    申请号：US16518807

    申请日：2019-07-22

    Applicant: VMware, Inc.

    Inventor： Dev Nag ,  Yanislav Yankov ,  Dongni Wang ,  Gregory T. Burk ,  Nicholas Mark Grant Stephen

    IPC: G06N20/00 ,  G06F9/54

    Abstract: The current document is directed to automated reinforcement-learning-based application managers that that are trained using adversarial training. During adversarial training, potentially disadvantageous next actions are selected for issuance by an automated reinforcement-learning-based application manager at a lower frequency than selection of next actions, according to a policy that is learned to provide optimal or near-optimal control over a computing environment that includes one or more applications controlled by the automated reinforcement-learning-based application manager. By selecting disadvantageous actions, the automated reinforcement-learning-based application manager is forced to explore a much larger subset of the system-state space during training, so that, upon completion of training, the automated reinforcement-learning-based application manager has learned a more robust and complete optimal or near-optimal control policy than had the automated reinforcement-learning-based application manager been trained by simulators or using management actions and computing-environment responses recorded during previous controlled operation of a computing-environment.

    COMPUTATIONALLY EFFICIENT REINFORCEMENT-LEARNING-BASED APPLICATION MANAGER
    12.
    发明申请
    COMPUTATIONALLY EFFICIENT REINFORCEMENT-LEARNING-BASED APPLICATION MANAGER 审中-公开

    公开(公告)号：US20200065156A1

    公开(公告)日：2020-02-27

    申请号：US16518717

    申请日：2019-07-22

    Applicant: VMware, Inc.

    Inventor： Dev Nag ,  Yanislav Yankov ,  Dongni Wang ,  Gregory T. Burk ,  Nicholas Mark Grant Stephen

    IPC: G06F9/50 ,  G06N20/00 ,  G06N3/02

    Abstract: The current document is directed to automated reinforcement-learning-based application managers that obtain increased computational efficiency by reusing learned models and by using human-management experience to truncate state and observation vectors. Learned models of managed environments that receive component-associated inputs can be partially or completely reused for similar environments. Human managers and administrators generally use only a subset of the available metrics in managing an application, and that subset can be used as an initial subset of metrics for learning an optimal or near-optimal control policy by an automated reinforcement-learning-based application manager.

    MODULAR REINFORCEMENT-LEARNING-BASED APPLICATION MANAGER
    13.
    发明申请
    MODULAR REINFORCEMENT-LEARNING-BASED APPLICATION MANAGER 审中-公开

    公开(公告)号：US20200065128A1

    公开(公告)日：2020-02-27

    申请号：US16261253

    申请日：2019-01-29

    Applicant: VMware, Inc.

    Inventor： Dev Nag ,  Gregory T. Burk ,  Janislav Jankov ,  Nick Stephen ,  Dongni Wang

    IPC: G06F9/455 ,  G06F9/50 ,  G06F9/48 ,  G06N20/00 ,  G06N3/08

    Abstract: The current document is directed to a modular reinforcement-learning-based application manager that can be deployed in various different computational environments without extensive modification and interface development. The currently disclosed modular reinforcement-learning-based application manager interfaces to observation and action adapters and metadata that provide a uniform and, in certain implementations, self-describing external interface to the various different computational environments which the modular reinforcement-learning-based application manager may be operated to control. In addition, certain implementations of the currently disclosed modular reinforcement-learning-based application manager interface to a user-specifiable reward-generation interface to allow the rewards that provide feedback from the computational environment to the modular reinforcement-learning-based application manager to be tailored to meet a variety of different user expectations and desired control policies.

    Simulator-training for automated reinforcement-learning-based application-managers
    14.
    发明授权
    Simulator-training for automated reinforcement-learning-based application-managers 有权

    公开(公告)号：US11238372B2

    公开(公告)日：2022-02-01

    申请号：US16518845

    申请日：2019-07-22

    Applicant: VMware, Inc.

    Inventor： Dev Nag ,  Yanislav Yankov ,  Dongni Wang ,  Gregory T. Burk ,  Nicholas Mark Grant Stephen

    IPC: G06N20/00 ,  G06K9/62 ,  G06N3/02 ,  G06F30/20

    Abstract: The current document is directed to methods and systems for simulation-based training of automated reinforcement-learning-based application managers. Simulators are generated from data collected from controlled computing environments controlled and may employ any of a variety of different machine-learning models to learn state-transition and reward models. The current disclosed methods and systems provide facilities for visualizing aspects of the models learned by a simulator and for initializing simulator models using domain information. In addition, the currently disclosed simulators employ weighted differences computed from simulator-generated and training-data state transitions for feedback to the machine-learning models to address various biases and deficiencies of commonly employed difference metrics in the context of training automated reinforcement-learning-based application managers.

    Automated reinforcement-learning-based application manager that learns and improves a reward function
    17.
    发明授权
    Automated reinforcement-learning-based application manager that learns and improves a reward function 有权

    公开(公告)号：US10963313B2

    公开(公告)日：2021-03-30

    申请号：US16518763

    申请日：2019-07-22

    Applicant: VMware, Inc.

    Inventor： Dev Nag ,  Yanislav Yankov ,  Dongni Wang ,  Gregory T. Burk ,  Nicholas Mark Grant Stephen

    IPC: G06N3/08 ,  G06F17/16 ,  G06F9/50 ,  G06N3/02 ,  G06N20/00

    Abstract: The current document is directed to automated reinforcement-learning-based application managers that learn and improve the reward function that steers reinforcement-learning-based systems towards optimal or near-optimal policies. Initially, when the automated reinforcement-learning-based application manager is first installed and launched, the automated reinforcement-learning-based application manager may rely on human-application-manager action inputs and resulting state/action trajectories to accumulate sufficient information to generate an initial reward function. During subsequent operation, when it is determined that the automated reinforcement-learning-based application manager is no longer following a policy consistent with the type of management desired by human application managers, the automated reinforcement-learning-based application manager may use accumulated trajectories to improve the reward function.

    TRANSFERABLE TRAINING FOR AUTOMATED REINFORCEMENT-LEARNING-BASED APPLICATION-MANAGERS
    20.
    发明申请
    TRANSFERABLE TRAINING FOR AUTOMATED REINFORCEMENT-LEARNING-BASED APPLICATION-MANAGERS 审中-公开

    公开(公告)号：US20200065670A1

    公开(公告)日：2020-02-27

    申请号：US16518831

    申请日：2019-07-22

    Applicant: VMware, Inc.

    Inventor： Dev Nag ,  Yanislav Yankov ,  Dongni Wang ,  Gregory T. Burk ,  Nicholas Mark Grant Stephen

    IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62 ,  G06F9/30

    Abstract: The current document is directed to transfer of training received by a first automated reinforcement-learning-based application manager while controlling a first application is transferred to a second automated reinforcement-learning-based application manager which controls a second application different from the first application. Transferable training provides a basis for automated generation of applications from application components. Transferable training is obtained from composition of applications from application components and composition of reinforcement-learning-based-control-and-learning constructs from reinforcement-learning-based-control-and-learning constructs of application components.

Patent Agency Ranking