Techniques are disclosed for restoring a modified document to an original state. The modified document is scanned into a digital form using an optical scanning device. The content of the modified digital document including one or more annotations is then grouped into several components, including text, images, form fields and text boxes, and marked shapes, based on corresponding component specifications. Each component is then categorized as being structured or unstructured. Structured components that correspond with representative entries in a component repository, such as text in a standard font size, weight and style, are identified as core document content. Unstructured components are identified as annotated document content or highlighted document content, depending on certain characteristics of the components. The categorized and identified components can then be presented separately or in various combinations.