A single-photon or ultra-weak light multi-D imaging spectral system and method. In order to realize rough time resolution, a time-resolved single-photon counting 2D imaging system for forming color or grey imaging is provided. Moreover, in order to realize high-precision time resolution, the system comprises a light source, an imaging spectral measurement unit, an electric detection unit, a system control unit and an algorithm unit. The light carrying information of an object is imaged on a spatial light modulator and randomly modulated according to compressed sensing theory, emergent light of a grating is collected using a point or array single-photon detector, the number of photons and photon arrival time are recorded, and reconstruction is carried out using the compressed sensing algorithm and related algorithm of the spectral imaging. The system provides single-photon detection sensitivity, high time resolution and wide spectral range, and can be applied in numerous new high-tech industries.