Automated regression detection framework for supporting robust version changes of machine learning applications

    公开(公告)号:US12093842B2

    公开(公告)日:2024-09-17

    申请号:US17008808

    申请日:2020-09-01

    申请人: SAP SE

    IPC分类号: G06N5/04 G06N20/00

    CPC分类号: G06N5/04 G06N20/00

    摘要: Methods, systems, and computer-readable storage media for receiving a project structure representing a regression test file directory for regression inference and including a set of test scenarios, determining that a test scenario of the set of test scenarios is to be executed, transmitting a request for a test inference job to be executed using a second version of the application, the test inference job representing the test scenario, receiving a set of actual results of the test inference job, calculating a prediction score based on the set of actual results and a set of expected results of the test scenario, and selectively indicating regression of the one or more ML models of the test scenario based on the prediction score.

    ADAPTIVE TRAINING COMPLETION TIME AND STATUS FOR MACHINE LEARNING MODELS

    公开(公告)号:US20230229961A1

    公开(公告)日:2023-07-20

    申请号:US17646889

    申请日:2022-01-04

    申请人: SAP SE

    IPC分类号: G06N20/00

    CPC分类号: G06N20/00

    摘要: Methods, systems, and computer-readable storage media for providing a set of heuristics representative of training data that is to be used to process a ML model through a training pipeline, the training pipeline including multiple phases, determining a set of time estimates by providing the set of heuristics as input to a training heuristics model that provides the set of time estimates as output, each time estimate in the set of time estimates indicating an estimated duration of a respective phase of the training pipeline, receiving, during processing of the ML model through the training pipeline, progress data representative of a progress of processing of the ML model, determining a set of status estimates including a status estimate for each phase of the training pipeline based on the progress data, and transmitting the set of time estimates and the set of status estimates for display.

    Automated regression detection system for robust enterprise machine learning applications

    公开(公告)号:US11681946B2

    公开(公告)日:2023-06-20

    申请号:US16408609

    申请日:2019-05-10

    申请人: SAP SE

    IPC分类号: G06N20/00

    CPC分类号: G06N20/00

    摘要: Methods, systems, and computer-readable storage media for determining, by an automated regression detection system (ARDS), that training of a ML model is complete, the ML model being a version of a previously trained ML model, and in response, automatically, by the ARDS: retrieving the ML model, executing regression testing and detection using the ML model, generating regression results relative to the previously trained ML model, and publishing the regression results.

    USER ACCEPTANCE TEST SYSTEM FOR MACHINE LEARNING SYSTEMS

    公开(公告)号:US20220300754A1

    公开(公告)日:2022-09-22

    申请号:US17203921

    申请日:2021-03-17

    申请人: SAP SE

    IPC分类号: G06K9/62 G06N20/00

    摘要: Methods, systems, and computer-readable storage media for receiving, by a ML application executing within a cloud platform, a first inference request, the first inference request including first inference data, transmitting, by the ML application, the first inference data to the UAT system within the cloud platform, retrieving, by the UAT system, a first ML model in response to the inference request, the first ML model being in an inactive state, providing, by the UAT system, a first inference based on the first inference data using the first ML model, providing a first accuracy evaluation at least partially based on the first inference, and transitioning the first ML model from the inactive state to an active state, the first ML model being used for production in the active state.

    AUTOMATED REGRESSION DETECTION FRAMEWORK FOR SUPPORTING ROBUST VERSION CHANGES OF MACHINE LEARNING APPLICATIONS

    公开(公告)号:US20220067548A1

    公开(公告)日:2022-03-03

    申请号:US17008808

    申请日:2020-09-01

    申请人: SAP SE

    IPC分类号: G06N5/04 G06N20/00

    摘要: Methods, systems, and computer-readable storage media for receiving a project structure representing a regression test file directory for regression inference and including a set of test scenarios, determining that a test scenario of the set of test scenarios is to be executed, transmitting a request for a test inference job to be executed using a second version of the application, the test inference job representing the test scenario, receiving a set of actual results of the test inference job, calculating a prediction score based on the set of actual results and a set of expected results of the test scenario, and selectively indicating regression of the one or more ML models of the test scenario based on the prediction score.