Described is a system and method for ephemeral betting within a panoramic video environment. A panoramic video environment is defined as a virtual experience, viewed on an information handling device (e.g., personal computers, mobile devices, virtual reality headsets, smart televisions, etc.) where the user views a panorama from a first person perspective relative to the camera capturing the video. The “view” may be a two-dimensional “portal” into the panoramic environment, or a stereoscopic view. The user will typically have directional control in the view portal, including the ability to pan, tilt, and zoom from the perspective of the camera. The described system and method creates wagering opportunities by combining the experience of viewing an event, simultaneously with other spectators, with real-time analytics and statistics derived from an object tracking system, thereby providing ephemeral wagering opportunities.