The described technology is generally directed towards techniques to measure interaction latency. Interaction latency can be measured at a client device by measuring time intervals between user inputs and corresponding feedback, such as video frames responsive to the user inputs. Feedback can comprise communication bursts received at the client device. The communication bursts can be detected and correlated with user inputs in order to measure interaction latency. Feedback can also comprise video response features which are responsive to the user inputs. Received video frames can be analyzed to discover response features, and video frames including the response features can be correlated with user inputs in order to measure interaction latency.