In implementations of augmented video prototyping, a mobile device records augmented video data as a captured video of a recorded scene in an environment, the augmented video data including augmented reality tracking data as 3D spatial information relative to objects in the recorded scene. A video prototyping module localizes the mobile device with reference to the objects in the recorded scene using the 3D spatial information for the mobile device being within boundaries of the recorded scene in the environment. The video prototyping module can generate an avatar for display that represents the mobile device at a current location from a perspective of the recorded scene, and create a spatial layer over a video frame at the current location of the avatar that represents the mobile device. The spatial layer is an interactive interface on which to create an augmented reality feature that displays during playback of the captured video.