公开(公告)号：US12236201B1

公开(公告)日：2025-02-25

申请号：US18677561

申请日：2024-05-29

Applicant: Snowflake Inc.

Inventor： Andrzej Szwabe

IPC: G06F40/40 ,  G06F40/284

Abstract: Examples provide enhanced machine learning model accuracy through post-hoc confidence score calibration. A machine learning (ML) system receives results generated by an ML model, the results comprising at least one confidence score and electronic documents. The ML system processes the results generated by the ML model comprising performing document understanding by extracting data points from the electronic documents. The ML system associates the confidence score with the extracted data points and calibrates a confidence score associated with the extracted data points using a post-hoc calibration solution set. The ML system implements confidence scoring recalibration comprising aligning the confidence score with prediction accuracy and adjusting the generated confidence score by the recalibration. Based on adjusting the confidence score, the ML system extracts an individual element of information from the electronic documents comprising an extracted value. The ML system generates an output comprising the extracted values and an adjusted confidence score.

公开(公告)号：US20250061287A1

公开(公告)日：2025-02-20

申请号：US18677561

申请日：2024-05-29

Applicant: Snowflake Inc.

Inventor： Andrzej Szwabe

IPC: G06F40/40 ,  G06F40/284

Abstract: Examples provide enhanced machine learning model accuracy through post-hoc confidence score calibration. A machine learning (ML) system receives results generated by an ML model, the results comprising at least one confidence score and electronic documents. The ML system processes the results generated by the ML model comprising performing document understanding by extracting data points from the electronic documents. The ML system associates the confidence score with the extracted data points and calibrates a confidence score associated with the extracted data points using a post-hoc calibration solution set. The ML system implements confidence scoring recalibration comprising aligning the confidence score with prediction accuracy and adjusting the generated confidence score by the recalibration. Based on adjusting the confidence score, the ML system extracts an individual element of information from the electronic documents comprising an extracted value. The ML system generates an output comprising the extracted values and an adjusted confidence score.

公开(公告)号：US12032919B1

公开(公告)日：2024-07-09

申请号：US18450700

申请日：2023-08-16

Applicant: Snowflake Inc.

Inventor： Andrzej Szwabe

IPC: G06F40/40 ,  G06F40/284

CPC classification number: G06F40/40 ,  G06F40/284

Abstract: Examples provide a large language model confidence scoring post-calibration based on a combination of temperature scaling, softmax denominator top-k probabilities selection, and polynomial regression. A secure machine learning system receives results generated by a machine learning (ML) model, the results including at least one confidence score. The secure ML system identifies at least one challenge in accuracy of the results generated by the ML model configured to perform document processing and understanding. The secure machine learning system implements confidence scoring recalibration to address at least one challenge, the confidence scoring recalibration including functionality to assess reliability of the results generated by the ML model, and applies post-processing calibration to the at least one confidence score generated by the confidence scoring recalibration to enhance performance of the ML model, the post-processing calibration including adjusting the at least one confidence score generated by the confidence scoring recalibration.