Techniques are described for performing a read scan process on a non-volatile memory system in order to determine memory blocks that may have a high bit error rate, where if such blocks are found they can be refreshed. Rather than work through the blocks of a memory system sequentially based on the physical block addresses, the memory system maintains a measure of data quality, such as an estimated or average bit error rate, for multi-block groups. For example, the groups can correspond to regions of memory die in the system. The groups are ranked by their data quality, with the groups being scanned in order of the data quality. The blocks within a group can also be ranked, based on factors such as the program/erase count.