A system and method that uses midservers located between the business enterprise computer infrastructure and the cloud-based infrastructure to collect, aggregate, analyze, transform, and securely transmit data from a multitude of computing devices and peripherals at an external network to a cloud-based service. The system and method make use of a plurality of virtual and physical worker agents which can be dynamically instantiated by a transformation engine to carry out one or more transformation sequences, based on pipeline instructions, to a received data stream to prepare the data for transmission as a target data stream format.