The silicone-based textile finishing agent comprises (a) an organopolysiloxane having at least one hydrocarbyl group substituted with a phosphate ester group --PO(OR).sub.2, in which R is a monovalent hydrocarbon group, (b) an alkoxy silane having at least one organofunctional group such as amino and epoxy groups in a molecule, and, optionally, (c) a catalyst capable of promoting the reaction between the components (a) and (b), preferably, in the form of an aqueous emulsion. The textile finishing agent is capable of imparting excellent softness and slipperiness to the textile materials such as waddings of polyester or acrylic fibers treated therewith without the disadvantage of yellowing by heat treatment.