Systems and methods are disclosed for handling read disturbs based on one or more characteristics of read operations performed on a non-volatile memory (“NVM”). In some embodiments, a control circuitry of a system can generate a variable damage value determined based on one or more characteristics of a read operation. Using the damage value, the control circuitry can update a score associated with the block. If the control circuitry determines that the score exceeds a pre-determined threshold, at least a portion of the block can be relocated to a different memory location in the NVM. In some embodiments, portions of the block may be relocated over a period of time.