公开(公告)号：US10002205B2

公开(公告)日：2018-06-19

申请号：US14947382

申请日：2015-11-20

Applicant: Oracle International Corporation

Inventor： Nicholas Roth ,  Sungpack Hong ,  Siegfried Depner ,  Thomas Manhardt ,  Hassan Chafi

IPC: G06F17/30

CPC classification number: G06F16/9024 ,  G06F16/278

Abstract: Techniques herein index data transferred during distributed graph processing. In an embodiment, a system of computers divides a directed graph into partitions. The system creates one partition per computer and distributes each partition to a computer. Each computer builds four edge lists that enumerate edges that connect the partition of the computer with a partition of a neighbor computer. Each of the four edge lists has edges of a direction, which may be inbound or outbound from the partition. Edge lists are sorted by identifier of the vertex that terminates or originates each edge. Each iteration of distributed graph analysis involves each computer processing its partition and exchanging edge data or vertex data with neighbor computers. Each computer uses an edge list to build a compactly described range of edges that connect to another partition. The computers exchange described ranges with their neighbors during each iteration.

公开(公告)号：US20170147706A1

公开(公告)日：2017-05-25

申请号：US14947382

申请日：2015-11-20

Applicant: Oracle International Corporation

Inventor： Nicholas Roth ,  Sungpack Hong ,  Siegfried Depner ,  Thomas Manhardt ,  Hassan Chafi

IPC: G06F17/30

CPC classification number: G06F17/30958 ,  G06F17/30584

Abstract: Techniques herein index data transferred during distributed graph processing. In an embodiment, a system of computers divides a directed graph into partitions. The system creates one partition per computer and distributes each partition to a computer. Each computer builds four edge lists that enumerate edges that connect the partition of the computer with a partition of a neighbor computer. Each of the four edge lists has edges of a direction, which may be inbound or outbound from the partition. Edge lists are sorted by identifier of the vertex that terminates or originates each edge. Each iteration of distributed graph analysis involves each computer processing its partition and exchanging edge data or vertex data with neighbor computers. Each computer uses an edge list to build a compactly described range of edges that connect to another partition. The computers exchange described ranges with their neighbors during each iteration.

公开(公告)号：US20190205178A1

公开(公告)日：2019-07-04

申请号：US16353050

申请日：2019-03-14

Applicant: ORACLE INTERNATIONAL CORPORATION

Inventor： Jinsu Lee ,  Sungpack Hong ,  Siegfried Depner ,  Nicholas Roth ,  Thomas Manhardt ,  Hassan Chafi

IPC: G06F9/50 ,  G06F9/54

CPC classification number: G06F9/5066 ,  G06F9/546

Abstract: Techniques herein provide job control and synchronization of distributed graph-processing jobs. In an embodiment, a computer system maintains an input queue of graph processing jobs. In response to de-queuing a graph processing job, a master thread partitions the graph processing job into distributed jobs. Each distributed job has a sequence of processing phases. The master thread sends each distributed job to a distributed processor. Each distributed job executes a first processing phase of its sequence of processing phases. To the master thread, the distributed job announces completion of its first processing phase. The master thread detects that all distributed jobs have announced finishing their first processing phase. The master thread broadcasts a notification to the distributed jobs that indicates that all distributed jobs have finished their first processing phase. Receiving that notification causes the distributed jobs to execute their second processing phase. Queues and barriers provide for faults and cancellation.

公开(公告)号：US20190171490A1

公开(公告)日：2019-06-06

申请号：US16270135

申请日：2019-02-07

Applicant: Oracle International Corporation

Inventor： Thomas Manhardt ,  Sungpack Hong ,  Siegfried Depner ,  Jinsu Lee ,  Nicholas Roth ,  Hassan Chafi

IPC: G06F9/50 ,  H04L29/08

Abstract: Techniques are provided for dynamically self-balancing communication and computation. In an embodiment, each partition of application data is stored on a respective computer of a cluster. The application is divided into distributed jobs, each of which corresponds to a partition. Each distributed job is hosted on the computer that hosts the corresponding data partition. Each computer divides its distributed job into computation tasks. Each computer has a pool of threads that execute the computation tasks. During execution, one computer receives a data access request from another computer. The data access request is executed by a thread of the pool. Threads of the pool are bimodal and may be repurposed between communication and computation, depending on workload. Each computer individually detects completion of its computation tasks. Each computer informs a central computer that its distributed job has finished. The central computer detects when all distributed jobs of the application have terminated.

公开(公告)号：US20170351551A1

公开(公告)日：2017-12-07

申请号：US15175920

申请日：2016-06-07

Applicant: Oracle International Corporation

Inventor： Thomas Manhardt ,  Sungpack Hong ,  Siegfried Depner ,  Jinsu Lee ,  Nicholas Roth ,  Hassan Chafi

IPC: G06F9/50

CPC classification number: G06F9/5061 ,  G06F9/5038 ,  G06F9/5066 ,  G06F9/5083 ,  G06F9/522 ,  H04L67/10

Abstract: Techniques are provided for dynamically self-balancing communication and computation. In an embodiment, each partition of application data is stored on a respective computer of a cluster. The application is divided into distributed jobs, each of which corresponds to a partition. Each distributed job is hosted on the computer that hosts the corresponding data partition. Each computer divides its distributed job into computation tasks. Each computer has a pool of threads that execute the computation tasks. During execution, one computer receives a data access request from another computer. The data access request is executed by a thread of the pool. Threads of the pool are bimodal and may be repurposed between communication and computation, depending on workload. Each computer individually detects completion of its computation tasks. Each computer informs a central computer that its distributed job has finished. The central computer detects when all distributed jobs of the application have terminated.

公开(公告)号：US10534657B2

公开(公告)日：2020-01-14

申请号：US15607985

申请日：2017-05-30

Applicant: Oracle International Corporation

Inventor： Siegfried Depner ,  Sungpack Hong ,  Thomas Manhardt ,  Jinsu Lee ,  Nicholas Roth ,  Hassan Chafi

IPC: G06F15/16 ,  G06F11/00

Abstract: Techniques minimize communication while loading a graph. In a distributed embodiment, each computer loads some edges of the graph. Each edge connects a source vertex (SV) to a destination vertex. For each SV of the edges, the computer hashes the SV to detect a tracking computer (TrC) that tracks on which computer does the SV reside. Each computer informs the TrC that the SV originates an edge that resides on that computer. For each SV, the TrC detects that the SV originates edges that reside on multiple providing computers (PCs). The TrC selects a target computer (TaC) from the multiple PCs to host the SV. The TrC instructs each PC, excluding the TaC, to transfer the SV and related edges that are connected to the SV to the TaC. A vertex's internal identifier indicates which computer hosts the vertex. The TrC maintains a mapping between external and internal identifiers.

公开(公告)号：US10318355B2

公开(公告)日：2019-06-11

申请号：US15413811

申请日：2017-01-24

Applicant: ORACLE INTERNATIONAL CORPORATION

Inventor： Jinsu Lee ,  Sungpack Hong ,  Siegfried Depner ,  Nicholas Roth ,  Thomas Manhardt ,  Hassan Chafi

IPC: G06F9/46 ,  G06F9/50 ,  G06F9/54

Abstract: Techniques herein provide job control and synchronization of distributed graph-processing jobs. In an embodiment, a computer system maintains an input queue of graph processing jobs. In response to de-queuing a graph processing job, a master thread partitions the graph processing job into distributed jobs. Each distributed job has a sequence of processing phases. The master thread sends each distributed job to a distributed processor. Each distributed job executes a first processing phase of its sequence of processing phases. To the master thread, the distributed job announces completion of its first processing phase. The master thread detects that all distributed jobs have announced finishing their first processing phase. The master thread broadcasts a notification to the distributed jobs that indicates that all distributed jobs have finished their first processing phase. Receiving that notification causes the distributed jobs to execute their second processing phase. Queues and barriers provide for faults and cancellation.

公开(公告)号：US20180352026A1

公开(公告)日：2018-12-06

申请号：US15607985

申请日：2017-05-30

Applicant: Oracle International Corporation

Inventor： Siegfried Depner ,  Sungpack Hong ,  Thomas Manhardt ,  Jinsu Lee ,  Nicholas Roth ,  Hassan Chafi

IPC: H04L29/08 ,  G06F9/38 ,  G06F3/00 ,  H04L9/06

Abstract: Techniques minimize communication while loading a graph. In a distributed embodiment, each computer loads some edges of the graph. Each edge connects a source vertex (SV) to a destination vertex. For each SV of the edges, the computer hashes the SV to detect a tracking computer (TrC) that tracks on which computer does the SV reside. Each computer informs the TrC that the SV originates an edge that resides on that computer. For each SV, the TrC detects that the SV originates edges that reside on multiple providing computers (PCs). The TrC selects a target computer (TaC) from the multiple PCs to host the SV. The TrC instructs each PC, excluding the TaC, to transfer the SV and related edges that are connected to the SV to the TaC. A vertex's internal identifier indicates which computer hosts the vertex. The TrC maintains a mapping between external and internal identifiers.

公开(公告)号：US11030014B2

公开(公告)日：2021-06-08

申请号：US16270135

申请日：2019-02-07

Applicant: Oracle International Corporation

Inventor： Thomas Manhardt ,  Sungpack Hong ,  Siegfried Depner ,  Jinsu Lee ,  Nicholas Roth ,  Hassan Chafi

IPC: G06F9/50 ,  H04L29/08 ,  G06F9/52 ,  G06F16/901 ,  G06F11/34

Abstract: Techniques are provided for dynamically self-balancing communication and computation. In an embodiment, each partition of application data is stored on a respective computer of a cluster. The application is divided into distributed jobs, each of which corresponds to a partition. Each distributed job is hosted on the computer that hosts the corresponding data partition. Each computer divides its distributed job into computation tasks. Each computer has a pool of threads that execute the computation tasks. During execution, one computer receives a data access request from another computer. The data access request is executed by a thread of the pool. Threads of the pool are bimodal and may be repurposed between communication and computation, depending on workload. Each computer individually detects completion of its computation tasks. Each computer informs a central computer that its distributed job has finished. The central computer detects when all distributed jobs of the application have terminated.

公开(公告)号：US10990595B2

公开(公告)日：2021-04-27

申请号：US16274210

申请日：2019-02-12

Applicant: Oracle International Corporation

Inventor： Nicholas Roth ,  Sungpack Hong ,  Petr Koupy ,  Jinsu Lee ,  Vasileios Trigonakis ,  Abderrahmane Melhaoui ,  Stefan Kaestle

IPC: G06F16/00 ,  G06F16/2453 ,  G06F16/901 ,  G06F16/27

Abstract: Techniques are described herein for asynchronous execution of queries on statically replicated graph data. In an embodiment, a graph is partitioned among a plurality of computers executing the graph querying engine. One or more high-degree vertices of the graph are each replicated in each graph partition. The partitions, including the replicated high-degree vertices, are loaded in memory of the plurality of computers. To execute a query, a query plan is generated based on the query. The query plan specifies a plurality of operators and an order for the plurality of operators. The order is such that if an operator requires data generated by another operator, then the other operator is ordered before the operator in the query plan. Replicated copies of a vertex is visited if matches made by subsequent operator(s) are limited by data unique to the replicated vertices.