公开(公告)号：US20210190364A1

公开(公告)日：2021-06-24

申请号：US16725961

申请日：2019-12-23

Applicant: Johnson Controls Technology Company

Inventor： Young M. Lee ,  Zhanhong Jiang ,  Viswanath Ramamurti ,  Sugumar Murugesan ,  Kirk H. Drees ,  Michael James Risbeck

IPC: F24F11/64 ,  F24F11/47

Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. Simulated experience data for the HVAC system is generated or received. The simulated experience data is used to initially train the RL model for HVAC control. The HVAC system operates within a building using the RL model and generates real experience data. A determination may be made to retrain the RL model. The real experience data is used to retrain the RL model. In some embodiments, both the simulated and real experience data are used to retrain the RL model. Experience data may be sampled according to various sampling functions. The RL model may be retrained multiple times over time. The RL model may be retrained less frequently over time as more real experience data is used to train the RL model.

公开(公告)号：US20190353379A1

公开(公告)日：2019-11-21

申请号：US15981181

申请日：2018-05-16

Applicant: Johnson Controls Technology Company

Inventor： Young M. Lee ,  Viswanath Ramamurti ,  Youngchoon Park ,  Sugumar Murugesan

IPC: F24F11/50 ,  G05B15/02 ,  G10L15/18 ,  G10L15/22 ,  F24F11/63 ,  G01S11/06

Abstract: A building management system includes one or more processors, and one or more computer-readable storage media communicably coupled to the one or more processors and having instructions stored thereon that cause the one or more processors to: receive utterance data from a voice assist device; determine a location of the voice assist device; analyze the utterance data to identify a sentiment relating to a temperature of the location; and control an HVAC system to adjust the temperature of the location based on the sentiment.

公开(公告)号：US20190353378A1

公开(公告)日：2019-11-21

申请号：US15980547

申请日：2018-05-15

Applicant: Johnson Controls Technology Company

Inventor： Viswanath Ramamurti ,  Young M. Lee ,  Youngchoon Park ,  Sugumar Murugesan

IPC: F24F11/50 ,  G05B13/02 ,  F24F11/65

Abstract: A building management system includes one or more processors, and one or more computer-readable storage media communicably coupled to the one or more processors and having instructions stored thereon that cause the one or more processors to: define a state of a zone or space within a building; control an HVAC system to adjust a temperature of the zone or space corresponding to a first action; receive utterance data from a voice assist device located in the zone or space; analyze the utterance data to identify a sentiment relating to the temperature of the zone or space; calculate a reward based on the state, the first action, and the sentiment; determine a second action to adjust the temperature of the zone or space based on the reward; and control the HVAC system to adjust the temperature of the zone or space corresponding to the second action.

公开(公告)号：US20210191342A1

公开(公告)日：2021-06-24

申请号：US16725999

申请日：2019-12-23

Applicant: Johnson Controls Technology Company

Inventor： Young M. Lee ,  Zhanhong Jiang ,  Viswanath Ramamurti ,  Sugumar Murugesan ,  Kirk H. Drees ,  Michael James Risbeck

IPC: G05B13/02 ,  G06N3/08 ,  G06N3/04 ,  G05B13/04 ,  F24F11/30 ,  F24F11/64

Abstract: Systems and methods for training a reinforcement learning (RL) model for HVAC control are disclosed herein. A calibrated simulation model is used to train a surrogate model of the HVAC system operating within a building. The surrogate model is used to generate simulated experience data for the HVAC system. The simulated experience data can be used to train a reinforcement learning (RL) model of the HVAC system. The RL model is used to control the HVAC system based on the current state of the system and the best predicted action to perform in the current state. The HVAC system generates real experience data based on the actual operation of the HVAC system within the building. The real experience data is used to retrain the surrogate model, and additional simulated experience data is generated using the surrogate model. The RL model can be retrained using the additional simulated experience data.

公开(公告)号：US10852023B2

公开(公告)日：2020-12-01

申请号：US15980547

申请日：2018-05-15

Applicant: Johnson Controls Technology Company

Inventor： Viswanath Ramamurti ,  Young M. Lee ,  Youngchoon Park ,  Sugumar Murugesan

IPC: F24F11/50 ,  F24F11/65 ,  G05B13/02 ,  F24F120/20 ,  F24F120/12

Abstract: A building management system includes one or more processors, and one or more computer-readable storage media communicably coupled to the one or more processors and having instructions stored thereon that cause the one or more processors to: define a state of a zone or space within a building; control an HVAC system to adjust a temperature of the zone or space corresponding to a first action; receive utterance data from a voice assist device located in the zone or space; analyze the utterance data to identify a sentiment relating to the temperature of the zone or space; calculate a reward based on the state, the first action, and the sentiment; determine a second action to adjust the temperature of the zone or space based on the reward; and control the HVAC system to adjust the temperature of the zone or space corresponding to the second action.