A retractable flagpole assembly includes a retractable flagpole, a top pivot holder rotatably capped on the top rod of the retractable flagpole and provided with a hanging lug for securing an inner top end of a flag, an end block spaced below the lower pivot holder and provided with a hanging lug for securing an inner bottom end of the flag, a link coupled between the top pivot holder and the end block, and a lower pivot holder rotatably sleeved onto the retractable flagpole and axially movable along the link and lockable to the end block.