JOB DISTRIBUTION WITHIN A GRID ENVIRONMENT
    2.
    发明申请

    公开(公告)号:US20190042309A1

    公开(公告)日:2019-02-07

    申请号:US16150163

    申请日:2018-10-02

    IPC分类号: G06F9/48 G06F9/50 H04L29/08

    摘要: According to one aspect of the present disclosure, a technique for job distribution within a grid environment includes receiving a job at a submission cluster for distribution of the job to at least one of a plurality of execution clusters where each execution cluster includes one or more execution hosts. Resource attributes are determined corresponding to each execution host of the execution clusters. For each execution cluster, execution hosts are grouped based on the resource attributes of the respective execution hosts. For each grouping of execution hosts, a mega-host is defined for the respective execution cluster where the mega-host for a respective execution cluster defines resource attributes based on the resource attributes of the respective grouped execution hosts. An optimum execution cluster is selected for receiving the job based on a weighting factor applied to select resources of the respective execution clusters.

    INDEPENDENT DATA PROCESSING ENVIRONMENTS WITHIN A BIG DATA CLUSTER SYSTEM

    公开(公告)号:US20170220667A1

    公开(公告)日:2017-08-03

    申请号:US15485952

    申请日:2017-04-12

    申请人: Databricks Inc.

    发明人: Ali Ghodsi Ion Stoica

    IPC分类号: G06F17/30 G06F9/50

    摘要: A cluster system includes an interface and a processor. The interface is to receive a request from a user associated with one of a plurality of shells. The processor is to determine a plurality of tasks to respond to the request; determine a local set of data and a shared set of data for a task of the plurality of tasks, wherein the local set of data is associated with the one of the plurality of shells; and provide the task, a local set indication, and a shared set indication to a worker associated with the task, wherein the local set indication refers to the local set of data and the shared set indication refers to the shared set of data.