Techniques are disclosed for allowing remote participation in collaborative video review based on joint state data, for a video collaboration session, maintained by a video collaboration service. A user at a participant client device may provide one or more annotations, such as a drawing annotation, for the video data via a client application. The client application transmits computer-readable instructions for re-creating the drawing annotation to the service, which distributes the drawing instructions to the other participant client devices. Using the drawing instructions, the client applications at the client devices are configured to re-create the drawing annotation on the associated video frame displayed at the client devices. The joint state data communicated to client devices for a given session may include co-presence data that efficiently increases communication among the participants in a session. Co-presence data received by a participant client device may be depicted in a GUI of the client application.