Techniques are disclosed that involve detecting and tracking groups of people associated with a user device (e.g., people watching television), and customizing the experience to the group. Various features may be employed. Such features may include classification of individuals, identification of commonly occurring groupings of people, and identification of the presence of group outsiders. Based on the presence of such individuals, groups, and/or outsiders, delivery of information to the user device may be controlled.