A mat has an embedded array of light sensors under a micro-faceted lens array to provide an array of light intensity values for various incident angles of incident real world light. Virtual objects may be projected onto the mat and, using the information from the light sensors, virtually illuminated to match the real-world lighting environment. A touch sensor array may be embedded in the mat to allow users to interact with the virtual objects and provide other input. The surface of the micro-faceted lens array may be coated with a patterned infrared (IR) reflective coating, such that the mat passively reflects a specific band of IR wavelengths in a pattern to act as a fiducial marker.