MULTI-DIMENSIONAL DATABASE PLATFORM INCLUDING AN APACHE KAFKA FRAMEWORK AND AN AUXILIARY DATABASE FOR EVENT DATA PROCESSING AND PROVISIONING

    公开(公告)号:US20230169043A1

    公开(公告)日:2023-06-01

    申请号:US17537784

    申请日:2021-11-30

    CPC classification number: G06F16/18

    Abstract: Systems and methods for event data processing and provisioning using a multi-dimensional database platform including an Apache Kafka framework and an auxiliary database are provided. Methods may include receiving an event profile from a data producer module, tagging the event profile with a timestamp, a unique event identification code, an event type descriptor, a data producer identifier, and a workflow identifier. The method may also include logging, in the Apache Kafka framework, the timestamp and the event identification code, as part of a sequential stream of events; and storing, in the auxiliary database, the event identification code, the event type descriptor, the data producer identifier, and the workflow identifier. The method may include receiving a request from a data consumer module for a customized workflow diagnostic and generating the customized workflow diagnostic based on the specification and on data from the Apache Kafka framework and the auxiliary database as linked via the event identification code. The method may also include transmitting the customized workflow diagnostic to the data consumer module.

    Multi-dimensional database platform including an apache kafka framework and an auxiliary database for event data processing and provisioning

    公开(公告)号:US11822519B2

    公开(公告)日:2023-11-21

    申请号:US17537784

    申请日:2021-11-30

    CPC classification number: G06F16/18

    Abstract: Systems and methods for event data processing and provisioning using a multi-dimensional database platform including an Apache Kafka framework and an auxiliary database are provided. Methods may include receiving an event profile from a data producer module, tagging the event profile with a timestamp, a unique event identification code, an event type descriptor, a data producer identifier, and a workflow identifier. The method may also include logging, in the Apache Kafka framework, the timestamp and the event identification code, as part of a sequential stream of events; and storing, in the auxiliary database, the event identification code, the event type descriptor, the data producer identifier, and the workflow identifier. The method may include receiving a request from a data consumer module for a customized workflow diagnostic and generating the customized workflow diagnostic based on the specification and on data from the Apache Kafka framework and the auxiliary database as linked via the event identification code. The method may also include transmitting the customized workflow diagnostic to the data consumer module.

    DELTA DATABASE DATA PROVISIONING
    5.
    发明申请

    公开(公告)号:US20230036517A1

    公开(公告)日:2023-02-02

    申请号:US17387120

    申请日:2021-07-28

    Abstract: A data exchange that provides historical data indexed by date is provided. The data exchange may include a raw data layer, a model data layer, a delta staging layer, a delta database and a plurality of workspaces. The raw data layer may be a landing zone for raw data records. The model data layer may include modeled data records. The delta staging layer may be a landing zone for changed data. The changed data may correspond to changes made to the data records. The delta database may be divided into partitions. Each partition may hold data records that changed during a given time period. A plurality of data records may be continuously transferred from the raw data layer to both the model data layer and the delta staging layer. Once, during a predetermined time period, the contents of the delta staging layer may replace the contents of a partition.

    Self-service data provisioning system

    公开(公告)号:US11562027B1

    公开(公告)日:2023-01-24

    申请号:US17387063

    申请日:2021-07-28

    Abstract: A data exchange that provides self-service data provisioning is provided. The data exchange may include a raw data layer, a model data layer, a plurality of workspaces and a testing environment. The raw data layer may be a landing zone for raw data records received from systems of record. The raw data layer may receive a plurality of raw data records, model and process the data records and transfer the data records to the model data layer. The model data layer may be a data layer that includes data modeled to data exchange specifications and enables queries to be executed on the data included in the model data layer. Each workspace may be allocated to a consumer. The consumer may query the plurality of data records within the model data layer. The testing environment may test scripts to ensure that the scripts conform to a predetermined set of testing specifications.

    Bulk data extract hybrid job processing

    公开(公告)号:US12061621B2

    公开(公告)日:2024-08-13

    申请号:US17387194

    申请日:2021-07-28

    CPC classification number: G06F16/254 G06F9/52 G06F16/215 G06F16/258 G06F16/902

    Abstract: Methods for hybrid job processing may include receiving raw data records stored within a plurality of tables from a plurality of systems of record at a raw data layer within a data exchange. Methods may include generating, based on a data model, a list of dependencies between the plurality of tables. Each table included in a second subset of the plurality of tables may be dependent on at least one table included in a first subset of the plurality of tables. Methods may include processing the first subset of the plurality of tables concurrently with one another. The processing includes modeling the raw data records and transmitting the modeled data records to the model data layer. Methods may include processing each table included in the second subset after completion of processing of the table included in the first subset from which the table in the second subset depends on.

    Delta database data provisioning
    8.
    发明授权

    公开(公告)号:US11663216B2

    公开(公告)日:2023-05-30

    申请号:US17387120

    申请日:2021-07-28

    CPC classification number: G06F16/24554 G06F16/219 G06F16/2282

    Abstract: A data exchange that provides historical data indexed by date is provided. The data exchange may include a raw data layer, a model data layer, a delta staging layer, a delta database and a plurality of workspaces. The raw data layer may be a landing zone for raw data records. The model data layer may include modeled data records. The delta staging layer may be a landing zone for changed data. The changed data may correspond to changes made to the data records. The delta database may be divided into partitions. Each partition may hold data records that changed during a given time period. A plurality of data records may be continuously transferred from the raw data layer to both the model data layer and the delta staging layer. Once, during a predetermined time period, the contents of the delta staging layer may replace the contents of a partition.

    BULK DATA EXTRACT HYBRID JOB PROCESSING

    公开(公告)号:US20230030208A1

    公开(公告)日:2023-02-02

    申请号:US17387194

    申请日:2021-07-28

    Abstract: Methods for hybrid job processing may include receiving raw data records stored within a plurality of tables from a plurality of systems of record at a raw data layer within a data exchange. Methods may include generating, based on a data model, a list of dependencies between the plurality of tables. Each table included in a second subset of the plurality of tables may be dependent on at least one table included in a first subset of the plurality of tables. Methods may include processing the first subset of the plurality of tables concurrently with one another. The processing includes modeling the raw data records and transmitting the modeled data records to the model data layer. Methods may include processing each table included in the second subset after completion of processing of the table included in the first subset from which the table in the second subset depends on.

    SELF-SERVICE DATA PROVISIONING SYSTEM

    公开(公告)号:US20230030189A1

    公开(公告)日:2023-02-02

    申请号:US17387063

    申请日:2021-07-28

    Abstract: A data exchange that provides self-service data provisioning is provided. The data exchange may include a raw data layer, a model data layer, a plurality of workspaces and a testing environment. The raw data layer may be a landing zone for raw data records received from systems of record. The raw data layer may receive a plurality of raw data records, model and process the data records and transfer the data records to the model data layer. The model data layer may be a data layer that includes data modeled to data exchange specifications and enables queries to be executed on the data included in the model data layer. Each workspace may be allocated to a consumer. The consumer may query the plurality of data records within the model data layer. The testing environment may test scripts to ensure that the scripts conform to a predetermined set of testing specifications.

Patent Agency Ranking