A system and method for generating activity summaries for users. User activity information is received by a user activity information receiver module. Once enough data is received and processed, the data is analyzes and segmented to determine and create an activity summary or story. Content is selected. The selection of the content includes selection of content items, such as multimedia items, e.g. pictures and videos. Secondary information, such as user activity information or location information is analyzed. A story is generated based on the selected content, including the selected content. The story is represented by a display of the selected media and other information associated with the media.