A building management system includes one or more processors, and one or more computer-readable storage media communicably coupled to the one or more processors and having instructions stored thereon that cause the one or more processors to: define a state of a zone or space within a building; control an HVAC system to adjust a temperature of the zone or space corresponding to a first action; receive utterance data from a voice assist device located in the zone or space; analyze the utterance data to identify a sentiment relating to the temperature of the zone or space; calculate a reward based on the state, the first action, and the sentiment; determine a second action to adjust the temperature of the zone or space based on the reward; and control the HVAC system to adjust the temperature of the zone or space corresponding to the second action.