Audience segmentation can be based on a viewing angle of a user viewing a video of a multi-angle viewing environment. During playback, a sequence of the user-controlled viewing angles of the video are recorded. The sequence represents the viewing angle of the user at a given point in time. Based on the sequences of several users, a predominant sequence of viewing angles of the video is determined. One or more audience segment tags are assigned to the predominant sequence of viewing angles. During subsequent playbacks of the video, the sequence(s) of user-controlled viewing angles of the video are recorded. The recorded sequence(s) of the subsequent user(s) are compared to the predominant sequence of viewing angles of the video, and the subsequent user(s) are assigned to an audience segment based on the comparison and the corresponding audience segment tags.