The present invention provides a method of manufacturing a composite material made of a rubber and a clay mineral dispersed uniformly therein. The method proceeds exchanging inorganic ions of the clay mineral with organic onium ions to organize the clay mineral; mixing the organized clay mineral and a process oil and/or a plasticizer; and mixing the rubber material with the mixture of the organized clay mineral and the process oil and/or the plasticizer. Process oil and/or plasticizer are intercalated into the organized clay mineral. As a result, the interlayer distance of the clay mineral is enlarged. The most favorable mode of the present invention is for a barrier material against water, gas and the like and for a rubber material required for an improved mechanical property.