A system for monitoring brick in a rotary kiln includes an infrared sensor and a computing system configured to: obtain a digital model of a brick layer of a rotary kiln having a plurality of bricks, wherein the digital model of the brick layer is based on a measured brick thickness correlated with a measured infrared temperature for each brick; obtain infrared data of the rotary kiln with the at least one infrared imaging sensor; determine the measured infrared temperature for each brick; determine a brick thickness of a first brick in the brick layer of the rotary kiln based on the measured infrared temperature assigned to the first brick with the digital model of the brick layer; and provide the brick thickness of the first brick in a brick thickness report.