A connector retaining device includes a first retaining frame and a similar second retaining frame. The first retaining frame includes two parallel first clamping plates, a first front opening at a front side thereof between the first clamping plates, a first rear plate at a rear side, a first side plate at one lateral side, and a first side opening at another lateral side. The first rear plate defines a first slot exposed at said another lateral side. The first clamping plates are configured for sandwiching a first connector. The first front opening is configured for allowing insertion of a second connector into the first retaining frame to engage with the first connector. The first side opening is configured for allowing insertion of the first connector into the first retaining frame. And the first slot is configured for allowing the first cable to extend and slide therethrough.