公开(公告)号：US20140047422A1

公开(公告)日：2014-02-13

申请号：US13923458

申请日：2013-06-21

Applicant: NEC Laboratories America, Inc.

Inventor： Nishkam Ravi ,  Abhishek Verma ,  Srimat T. Chakradhar

IPC: G06F9/45

CPC classification number: G06F8/443 ,  G06F9/52 ,  G06F9/546

Abstract: Various methods are provided directed to a compiler-guided software accelerator for iterative HADOOP jobs. A method includes identifying intermediate data, generated by an iterative HADOOP application, below a predetermined threshold size and used less than a predetermined threshold time period. The intermediate data is stored in a memory device. The method further includes minimizing input, output, and synchronization overhead for the intermediate data by selectively using at any given time any one of a Message Passing Interface and Distributed File System as a communication layer. The Message Passing Interface is co-located with the HADOOP Distributed File System.

Abstract translation: 针对迭代HADOOP作业的编译器引导软件加速器提供了各种方法。 一种方法包括将由迭代HADOOP应用生成的中间数据识别为低于预定阈值大小并且使用小于预定阈值时间段的中间数据。 中间数据存储在存储设备中。 该方法还包括通过在任何给定时间选择性地使用消息传递接口和分布式文件系统中的任何一个作为通信层来最小化中间数据的输入，输出和同步开销。 消息传递接口与HADOOP分布式文件系统位于同一位置。

公开(公告)号：US09201638B2

公开(公告)日：2015-12-01

申请号：US13923458

申请日：2013-06-21

Applicant: NEC Laboratories America, Inc.

Inventor： Nishkam Ravi ,  Abhishek Verma ,  Srimat T. Chakradhar

IPC: G06F9/45 ,  G06F9/52 ,  G06F9/54

CPC classification number: G06F8/443 ,  G06F9/52 ,  G06F9/546

Abstract: Various methods are provided directed to a compiler-guided software accelerator for iterative HADOOP® jobs. A method includes identifying intermediate data, generated by an iterative HADOOP® application, below a predetermined threshold size and used less than a predetermined threshold time period. The intermediate data is stored in a memory device. The method further includes minimizing input, output, and synchronization overhead for the intermediate data by selectively using at any given time any one of a Message Passing Interface and Distributed File System as a communication layer. The Message Passing Interface is co-located with the HADOOP® Distributed File System.

Abstract translation: 针对迭代HADOOP®作业的编译器引导软件加速器提供了各种方法。 一种方法包括将迭代HADOOP应用产生的中间数据识别为低于预定阈值大小并且使用小于预定阈值时间段的中间数据。 中间数据存储在存储设备中。 该方法还包括通过在任何给定时间选择性地使用消息传递接口和分布式文件系统中的任何一个作为通信层来最小化中间数据的输入，输出和同步开销。 消息传递接口与HADOOP®分布式文件系统位于同一位置。