A housing of a portable electronic device includes a first frame unit, a second frame unit, and a panel unit. The first frame unit includes a plurality of retaining members. The second frame unit includes a plurality of fastening members. The panel unit includes a plurality of holding members corresponding to the retaining members and a plurality of assembling members corresponding to the fastening member. The retaining members and the holding members are correspondingly assembled together to assemble the first frame unit to the panel unit, and the fastening members and the assembling members are correspondingly assembled together to assemble the second frame unit to the panel unit.