Embodiments disclosed herein relate to systems and methods for performing fitting of an auditory prosthesis using tactile responses. A tactile feedback device determines a physical manipulation in response to a test stimulus. A type of adjustment can be determined based upon the type of the physical manipulation and the type of the test signal. A scaling of the adjustment can be determined based on the degree of the physical manipulation.