A resilient support column for height-adjustable chairs or the like includes a piston-and-cylinder type pneumatic spring that is installable in and removable from the column by means of a retaining sleeve structure carried by the spring. The retaining sleeve is connected to the column through a threaded connection that permits quick one-step insertion of the combined sleeve-spring unit into the column, while thereafter affording the advantage of threaded adjustment within the column or threaded removal therefrom. A simple mechanical fastener holds the retaining sleeve in place on the pneumatic spring during shipment and/or assembly.