Invention Grant
- Patent Title: Management of a data processing pipeline
-
Application No.: US15275307Application Date: 2016-09-23
-
Publication No.: US10346366B1Publication Date: 2019-07-09
- Inventor: Christopher Richard Jacques de Kadt , Benjamin Warren Mercier , Carlos Vara Callau , Timothy Daniel Cole , Aaron Gifford Freshwater , Sayantan Chakravorty , Allan Henry Vermeulen
- Applicant: Amazon Technologies, Inc.
- Applicant Address: US WA Seattle
- Assignee: Amazon Technologies, Inc.
- Current Assignee: Amazon Technologies, Inc.
- Current Assignee Address: US WA Seattle
- Agency: Davis Wright Tremaine LLP
- Main IPC: G06F16/18
- IPC: G06F16/18 ; G06F16/23 ; H04L12/18

Abstract:
A pipeline management service improves the operation of a processing pipeline by monitoring the operation of one or more journal services within the processing pipeline, and adjusting operational parameters of the one or more journal services. In various examples, the pipeline management service determines a model of the processing pipeline by identifying the producers and consumers associated with each of the one or more journal services, and determines a processing flow through the one or more journal services based at least in part on the identities of the associated producers and consumers. The pipeline management service acquires backlog information from each of the one or more journal services, and may adjust the operational parameters of a journal service associated with an identified processing bottleneck. In some examples, the pipeline management service adjusts capacity and reliability parameters of the one or more journal services to improve the operation of the processing pipeline.
Information query