A handle structure includes a gripping portion and a hinge portion. The gripping portion includes a handle and two levers. Each lever has a first hole, a second hole and a groove adjacent to the second hole. The handle is inserted into those second holes by pressing the handle toward the grooves. The hinge portion includes two flexible clamps. Each flexible clamp includes two protrusions against each other. The end with the first hole of the gripping portion presses the two protrusions to move apart from each other. Then, the protrusions are inserted in the first holes, and the gripping portion can swing around the hinge portion.