A heating system that reduces the cycling losses of a heat engine driven heat pump by storing rejected heat from the engine and using this rejected heat to meet a heating demand between periods of operation of the engine and heat pump so that over an extended time fewer start/stop cycles of the engine and heat pump are necessary. In a preferred embodiment a conventional storage-type hot water heater is used to store rejected heat, provide potable hot water, supply heat in the event of failure of the heat pump, supplement heat in periods of unusual demand, and provide low level heating requirements when heat pump operation would not be economical.