The invention discloses a steel structure connecting device applying prestress; the device consists of prestressed pull rod pairs, combined gusset plates, shear keys, steel columns and steel girders. The steel girders are vertically placed at the upper end of the steel columns to webplates being arranged in the length direction of the steel girders; combined gusset plates are fixedly connected at the external sides of two flanges at the upper tips of the steel columns; the prestressed pull rod pairs penetrate the upper flanges and the lower flanges of the steel girders and holes arranged on the combined gusset plates, one end of each prestressed pull rod pair is connected at the upper flanges of the steel girders, and the other end is connected on the combined gusset plates; the shear keys are connected with the webs and the lower flanges of the steel girders. When in installation, the prestressed pull rod pairs are firstly installed, then the prestress is applied, and the shear keys are finally installed. The prestressed steel pull rods which are arranged in pairs are adopted to resist bending and the shear keys are adopted to resist shearing, the structure is clear, the construction is convenient, the materials are saved, and the connecting device is especially suitable for monolayer lightweight steel constructions.