An excellently pleasant feeling of softness and elastic resilience can be imparted to a fabric material treated with the inventive fabric finishing agent which is an aqueous emulsion of a copolymer obtained by the emulsion polymerization, in an aqueous emulsion of an organopolysiloxane having radical-polymerizable groups, e.g., vinyl groups, or mercapto-substituted hydrocarbon groups, e.g., 3-mercaptopropyl groups, as the polymerization medium of an acrylic or methacrylic monomer, e.g., alkyl acrylates and methacrylates, in a specified amount relative to the amount of the organopolysiloxane. The acrylic or methacrylic monomer may be partly replaced with a polyfunctional monomer, e.g., acrylic and methacrylic acids, N-methylol acrylamide, glycidyl methacrylate, 2-hydroxyethyl methacrylate and the like, and/or a monomer of the third class, e.g., styrene and acrylonitrile. The emulsion may be further admixed with an organohydrogen polysiloxane as a crosslinking agent and a catalyst for the crosslinking reaction.