The technology disclosed herein pertains to a system and method for managing off-track retry. An implementation of a method of determining offset direction for read off-track retry includes storing analog to digital converter (ADC) values of data read from a data sector by a data reader in a read channel buffer, calculating an indicator value of the distribution of the ADC values, determining an amount of offset for the data reader based on the indicator value, and moving the data reader by the amount of offset before performing a read retry operation.