A device for attaching a bag to a luggage carrier of a bicycle includes a beam secured to the bag, two hooks installed on the beam to hook the luggage carrier and a restraining unit. The restraining unit includes a plate formed on the beam, a cover attached to the plate, at least one latch movably installed on the plate, at least one elastic element compressed between the cover and the latch so that an end of the latch is exposed from the plate for restraining the luggage carrier, a shackle movable on the plate between a first position for keeping the end of the latch exposed from the plate and a second position for allowing the retreat of the end of the latch into the plate and a elastic element compressed between the cover and the shackle for keeping the shackle in the first position.