公开(公告)号：US20220299233A1

公开(公告)日：2022-09-22

申请号：US17204605

申请日：2021-03-17

Applicant: Johnson Controls Technology Company

Inventor： Michael James Risbeck

IPC: F24F11/70 ,  G05B13/02 ,  G06N3/08 ,  G06K9/62

Abstract: A method for controlling temperature in a building zone to increase comfort and energy efficiency is shown. The method includes receiving historical data, the historical data indicative of the temperature and occupancy of the building zone during one or more historical states. The method includes training a system model to represent a dynamic response of the building zone based on the historical data. The method includes determining a control law by optimizing a policy function implemented as a neural network configured to process the trained system model. The method includes performing online control of the building zone using the control law.

公开(公告)号：US20210190364A1

公开(公告)日：2021-06-24

申请号：US16725961

申请日：2019-12-23

Applicant: Johnson Controls Technology Company

Inventor： Young M. Lee ,  Zhanhong Jiang ,  Viswanath Ramamurti ,  Sugumar Murugesan ,  Kirk H. Drees ,  Michael James Risbeck

IPC: F24F11/64 ,  F24F11/47

Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. Simulated experience data for the HVAC system is generated or received. The simulated experience data is used to initially train the RL model for HVAC control. The HVAC system operates within a building using the RL model and generates real experience data. A determination may be made to retrain the RL model. The real experience data is used to retrain the RL model. In some embodiments, both the simulated and real experience data are used to retrain the RL model. Experience data may be sampled according to various sampling functions. The RL model may be retrained multiple times over time. The RL model may be retrained less frequently over time as more real experience data is used to train the RL model.

公开(公告)号：US20210191342A1

公开(公告)日：2021-06-24

申请号：US16725999

申请日：2019-12-23

Applicant: Johnson Controls Technology Company

Inventor： Young M. Lee ,  Zhanhong Jiang ,  Viswanath Ramamurti ,  Sugumar Murugesan ,  Kirk H. Drees ,  Michael James Risbeck

IPC: G05B13/02 ,  G06N3/08 ,  G06N3/04 ,  G05B13/04 ,  F24F11/30 ,  F24F11/64

Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. A calibrated simulation model is used to train a surrogate model of the HVAC system operating within a building. The surrogate model is used to generate simulated experience data for the HVAC system. The simulated experience data can be used to train a reinforcement learning (RL) model of the HVAC system. The RL model is used to control the HVAC system based on the current state of the system and the best predicted action to perform in the current state. The HVAC system generates real experience data based on the actual operation of the HVAC system within the building. The real experience data is used to retrain the surrogate model, and additional simulated experience data is generated using the surrogate model. The RL model can be retrained using the additional simulated experience data.