公开(公告)号：US12254385B2

公开(公告)日：2025-03-18

申请号：US17389558

申请日：2021-07-30

Applicant: Tsinghua University

Inventor： Wenchuan Wu ,  Haotian Liu ,  Hongbin Sun ,  Bin Wang ,  Qinglai Guo

IPC: G06N20/00

Abstract: A method for multi-time scale reactive voltage control based on reinforcement learning in a power distribution network is provided, which relates to the field of power system operation and control. The method includes: constituting an optimization model for multi-time scale reactive voltage control in a power distribution network based on a reactive voltage control object of a slow discrete device and a reactive voltage control object of a fast continuous device in the power distribution network; constructing a hierarchical interaction training framework based on a two-layer Markov decision process based on the model; setting a slow agent for the slow discrete device and setting a fast agent for the fast continuous device; and deciding action values of the controlled devices by each agent based on measurement information inputted, so as to realize the multi-time scale reactive voltage control while the slow agent and the fast agent perform continuous online learning.

公开(公告)号：US11689021B2

公开(公告)日：2023-06-27

申请号：US17025154

申请日：2020-09-18

Applicant: TSINGHUA UNIVERSITY

Inventor： Wenchuan Wu ,  Haotian Liu ,  Hongbin Sun ,  Bin Wang ,  Qinglai Guo ,  Tian Xia

IPC: H02J3/18 ,  G06F30/20

CPC classification number: H02J3/18 ,  G06F30/20 ,  H02J2203/20

Abstract: A power grid reactive voltage control model training method. The method comprises: establishing a power grid simulation model; establishing a reactive voltage optimization model, according to a power grid reactive voltage control target; building interactive training environment based on Adversarial Markov Decision Process, in combination with the power grid simulation model and the reactive voltage optimization model; training the power grid reactive voltage control model through a joint adversarial training algorithm; and transferring the trained power grid reactive voltage control model to an online system. The power grid reactive voltage control model trained by using the method according to the present disclosure has transferability as compared with the traditional method, and may be directly used for online power grid reactive voltage control.

公开(公告)号：US11442420B2

公开(公告)日：2022-09-13

申请号：US17026364

申请日：2020-09-21

Applicant: TSINGHUA UNIVERSITY

Inventor： Wenchuan Wu ,  Haotian Liu ,  Hongbin Sun ,  Bin Wang ,  Qinglai Guo ,  Tian Xia

IPC: G05B19/00 ,  G05B19/042 ,  G06N20/00

Abstract: A power grid reactive voltage control method and control system based on two-stage deep reinforcement learning, comprising steps of: building interactive training environment based on Markov decision process, according to a regional power grid simulation model and a reactive voltage optimization model; training a reactive voltage control model offline by using a SAC algorithm, in the interactive training environment based on Markov decision process; deploying the reactive voltage control model to a regional power grid online system; and acquiring operating state information of the regional power grid, updating the reactive voltage control model, and generating an optimal reactive voltage control policy. As compared with the existing power grid optimizing method based on reinforcement learning, the online control training according to the present disclosure has costs and safety hazards greatly reduced, and is more suitable for deployment in an actual power system.

公开(公告)号：US20210356923A1

公开(公告)日：2021-11-18

申请号：US17026364

申请日：2020-09-21

Applicant: TSINGHUA UNIVERSITY

Inventor： Wenchuan Wu ,  Haotian Liu ,  Hongbin Sun ,  Bin Wang ,  Qinglai Guo ,  Tian Xia

IPC: G05B19/042 ,  G06N20/00

Abstract: A power grid reactive voltage control method and control system based on two-stage deep reinforcement learning, comprising steps of: building interactive training environment based on Markov decision process, according to a regional power grid simulation model and a reactive voltage optimization model; training a reactive voltage control model offline by using a SAC algorithm, in the interactive training environment based on Markov decision process; deploying the reactive voltage control model to a regional power grid online system; and acquiring operating state information of the regional power grid, updating the reactive voltage control model, and generating an optimal reactive voltage control policy. As compared with the existing power grid optimizing method based on reinforcement learning, the online control training according to the present disclosure has costs and safety hazards greatly reduced, and is more suitable for deployment in an actual power system.