Group management of video device playback provides a mechanism for shared playback of recorded video content. Digital video playback devices are linked to a central management system through network connections which manages the shared playback, or a peer-to-peer network connection is used without the central management system. One user may be responsible for all actions affecting content playback, or all users may be permitted to affect content playback, with certain actions of those users limited in type or quantity. Group membership and group member permissions can be accomplished using the digital playback device through interactive interfaces implemented by the digital video playback device, or can be accomplished using a personal computer coupled to the device or central management system. Users may comment on shared playback using interactive video-based commentary functionality that may provide a text channel or graphical images that may be selected from a set of pre-configured commentary items.