A common infrastructure collects diverse data and information from large numbers of mobile devices and traditional sensors at Internet scale to support multiple different applications simultaneously. The infrastructure includes a backend phenomenon layer that provides high level abstractions to applications such that they can express their data and information needs in a declarative fashion and coordinate the data collection and processing activities for all applications. An edge layer that manages devices, receives collection requirements from the backend layer, configures and instructs devices for data collection, and conducts aggregation and primitive processing of the data. This layer contains network edge nodes, such as base stations in a cellular network. Each node manages a set of local data generating networked devices. The device agent data layer using common agents on the data generating networked devices receives data collection instructions from the edge layer, performs data collection.