公开(公告)号：US20250111220A1

公开(公告)日：2025-04-03

申请号：US18374905

申请日：2023-09-29

Applicant: Amazon Technologies, Inc.

Inventor： Robert JONES ,  Aaron Robert BRADLEY ,  Leah Corene DANIELS ,  Leonardo Mendonça DE MOURA

IPC: G06N3/08 ,  G06N3/0475

Abstract: Generative pre-trained large language models (LLMs) can create domain-specific text answers in various formats like JSON, XML, HTML, SQL, or programming languages. However, LLMs may “hallucinate,” generating incorrect or nonsensical answers that diverge from reality, thus eroding trust in their outputs or worse. Disclosed techniques use a sampling-based approach and an equivalence checker. Multiple answers (samples) to a prompt are generated by the LLM; if they are equivalent, the LLM is likely answering correctly. If the samples disagree or contradict, it's more likely that the LLM is hallucinating, or the prompt is ambiguous. An automated reasoning equivalence checker is utilized to verify the samples' functional equivalency, providing a method to detect and possibly rectify hallucination issues in LLM-generated answers.