公开(公告)号：US11586911B2

公开(公告)日：2023-02-21

申请号：US16609330

申请日：2018-05-07

申请人： Telefonaktiebolaget LM Ericsson (publ)

发明人： Wei Huang ,  Wenfeng Hu ,  Tobias Ley ,  Martha Vlachou-Konchylaki

IPC分类号： G06N3/00 ,  G06N3/08 ,  G06N3/04 ,  G06N3/10 ,  G06N3/084

摘要： A pre-training apparatus and method for reinforcement learning based on a Generative Adversarial Network (GAN) is provided. GAN includes a generator and a discriminator. The method comprising receiving training data from a real environment where the training data includes a data slice corresponding to a first state-reward pair and a first state-action pair, training the GAN using the training data, training a relations network to extract a latent relationship of the first state-action pair with the first state-reward pair in a reinforcement learning context, causing the generator trained with training data to generate first synthetic data, processing a portion of the first synthetic data in the relations network to generate a resulting data slice, merging the second state-action pair portion of the first synthetic data with the second state-reward pair from the relations network to generate second synthetic data to update a policy for interaction with the real environment.

公开(公告)号：US11962475B2

公开(公告)日：2024-04-16

申请号：US17794176

申请日：2020-02-03

申请人： Telefonaktiebolaget LM Ericsson (publ)

发明人： Martha Vlachou-Konchylaki ,  Efthymios Stathakis ,  Arthur Gusmao ,  Jörg Niemöller ,  Swarup Kumar Mohalik ,  Yu Wang

IPC分类号： H04L41/14 ,  H04L41/147 ,  H04L41/16

CPC分类号： H04L41/145 ,  H04L41/147 ,  H04L41/16

摘要： The properties of a plurality of operational units are estimated by generating a central system state graph model representing the properties of the plurality of operational units as probabilities of transitions between states for the plurality of operational units, where the states represent operational data. Then a respective updated system state graph model is generated for each of the plurality of operational units, based on the central system state graph model and based on new operational data for the respective operational unit. A distance measure is determined between the respective updated system state graph models. If the distance measure fulfils a divergence criterion, a plurality of new central system state graph models are generated, each representing the properties of a respective subset of the plurality of operational units as the probabilities of transitions between states for the respective subset of the plurality of operational units.

公开(公告)号：US11177996B2

公开(公告)日：2021-11-16

申请号：US16498680

申请日：2017-04-04

申请人： Telefonaktiebolaget LM Ericsson (publ)

发明人： Vincent Huang ,  Roberta Daidone ,  Tobias Ley ,  Martha Vlachou-Konchylaki

IPC分类号： G06F15/173 ,  H04L12/24 ,  G06N20/00 ,  G06N7/00 ,  H04L12/46 ,  H04L29/08

摘要： A method for training a Software Agent to control a communication network is disclosed. The method comprises initialising a training network slice within the communication network, instantiating within the training network slice a cloned version of at least one Virtualised Network Function (VNF) comprised within a production network slice, mirroring traffic incoming to the VNFs of the production network slice and forwarding the mirrored traffic to the training network slice. The method further comprises causing a training instance of the Software Agent to execute a Reinforcement Learning algorithm on the training network slice, and transferring knowledge acquired by the training instance of the Software Agent to a production instance of the Software Agent.
Also disclosed are an apparatus and a computer program configured to carry out methods for training a Software Agent to control a communication network.