Disclosed herein are various embodiments for an augmented reality interaction, modeling, and annotation system. An embodiment operates by receiving an image including unknown data in an unknown format, including pixels. Each of the pixels is classified as one of a background pixel, a key pixel, or a value pixel representing the unknown data. For a plurality of the pixels classified as key pixels or value pixels, a plurality of locational data values associated with the unknown format are generated. Based on the locational data values, a key image and a corresponding value image from the received image are identified. The key image and the corresponding value image are output.