Desktop surface references are selected and applied to define a coordinate system for calibrating projected visual images and end user inputs at the projected visual images. For example, a desktop edge is detected with the depth camera by the increase in detected distance along the axis from the depth camera to the desktop edge, and then the desktop edge is used as an origin for a coordinate system that defines a projection area for presenting a user interface. Monitoring end user inputs and projected outputs relative to the desktop edge aids in coordinating interactions by a user through the projected user interface in the event the camera or projector move relative to the desktop surface.