A system and method for facilitating remote care management involving a patient having an implantable medical device (IMD). Upon establishing a remote care session between a patient controller device and a clinician programmer, wherein the clinician and the patient are remotely located with respect to each other, input from the patient or the clinician may be received via a user interface control associated with a particular functionality or aspect of the remote care session, including audiovisual (AV) communications, remote therapy programming, and related context. Responsive to the user input, a dialog interface is effectuated at one of the patient controller device and/or the clinician programmer. A user characterization label is received via the dialog interface from the user, wherein the user characterization label is indicative of a subjective assessment of the particular functionality of the remote care session, which may be used in generating user-labeled data pertaining thereto.