Various embodiments described herein provide for copying (e.g., to cache) a portion of defect management data for a block of a memory device, such as a non-volatile memory device of a memory sub-system, based on activity of the memory device. For instance, the portion of defect management data can be copied from a first-type memory device of the memory sub-system to a second-type memory device of the memory sub-system, where the first-type memory device stores defect management data for a working set of blocks of the non-volatile memory device being operated upon by the memory sub-system, where the second-type memory device is used to store defect management data for an active block of the working set of blocks, and where the second-type memory device has a faster access (e.g., read or write access) than the first-type memory device.