Abstract:
A system includes a processor and a non-transitory computer-readable medium. The non-transitory computer-readable medium comprises instructions executable by the processor to cause the system to perform a method. The method comprises receiving a first job to execute and executing the first job. A plurality of data associated with the first job is determined The plurality of data comprises data associated with (i) a second job executed immediately prior to the first job, (ii) a third job executed immediately after the first job, (iii) a determination of whether the first job failed or executed successfully and (iv) a type of data associated with the first job. The determined plurality of data is stored.
Abstract:
Embodiments of the disclosure can include MapReduce systems and methods with integral mapper and reducer compute runtime environments. An example system with an integral reducer compute runtime environment can include mappers and reducers executable on a computer cluster. The mappers can be operable to receive raw input data and generate first input data based on the raw input data. The mappers can be operable to generate first result data based on the first input data. Based on the first result data, the mappers can be operable to generate (K, V) pairs. The reducers can be operable to receive the (K, V) pairs and generate second input data based on the (K, V) pairs. The reducers can be operable to transmit the second input data to integral compute runtime environment being run within the reducers and operable to generate second result data based on the second input data. Based on the second result data, the reducers can be operable to generate output data.
Abstract:
A system includes a processor and a non-transitory computer-readable medium. The non-transitory computer-readable medium comprises instructions executable by the processor to cause the system to perform a method. The method comprises receiving a first job to execute and executing the first job. A plurality of data associated with the first job is determined. The plurality of data comprises data associated with (i) a second job executed immediately prior to the first job, (ii) a third job executed immediately after the first job, (iii) a determination of whether the first job failed or executed successfully and (iv) a type of data associated with the first job. The determined plurality of data is stored.