Examples are disclosed that relate to dynamically controlling light sources on an optically trackable peripheral device. One disclosed example provides a near-eye display device comprising an image sensor, a communications subsystem, a logic subsystem, and a storage subsystem. The storage subsystem stores instructions executable by the logic subsystem to control a peripheral device comprising a plurality of light sources by receiving image data from the image sensor, identifying in the image data a constellation of light sources formed by a subset of light sources of the peripheral device, and based upon the constellation of light sources identified, send to the peripheral device via the communications subsystem constellation information related to the constellation of light sources identified.