A retractable column structure includes a metal outer tubular member, a retaining bracket, and an inner tubular member. The outer tubular member includes a wall having outer, inner and end surfaces. The outer tubular member includes slots extending from the end surface to the inner surface. A rear end of each slot includes a hook protruding out of the inner surface. The retaining bracket includes a frame member abutted against the wall and locking hooks inserted into the inner surface and locked onto each corresponding hook. The inner tubular member moveably penetrates into the retaining bracket disposed between the tubular members and is concealed inside the outer tubular member. Accordingly, excellent support of inner and outer tubular members can be achieved for stable sliding therebetween.