The invention provides a rubber roller type automatic oil tea fruit harvester. The rubber roller type automatic oil tea fruit harvester comprises a chassis rack, a walking system, a picking head, a picking mechanism, a vision control mechanism, a hydraulic mechanism and a collection mechanism, wherein the walking system and the vision control mechanism are mounted on the chassis rack respectively; one end of the picking mechanism is connected with the picking head and the other end of the picking mechanism is fixed on the chassis rack; the walking system is connected with the hydraulic mechanism; the hydraulic mechanism is connected with the picking mechanism and the vision control mechanism respectively; and the collection mechanism is mounted in the chassis rack. According to the oil tea harvester based on vision control, a CCD (Charge Coupled Device) camera is used for acquiring oil tea trees at fixed time and identified oil tea fruits are continuously and automatically pickled and collected; meanwhile, the rubber roller type picking head can realize picking of oil tea fruits in a relatively large range in one step; the harvester has a relatively high automatic picking degree and good adaptability on a picking environment; the coordination performance of the picking mechanisms is excellent and the continuity of picking actions can be guaranteed; the picking efficiency is high and the unpicking rate is low; and the harvester particularly has low damage degree on oil tea buds and branches and leaves.