AUTOMATED EVALUATION OF EVIDENCE MAPPING MODELS

    公开(公告)号:US20250005063A1

    公开(公告)日:2025-01-02

    申请号:US18344739

    申请日:2023-06-29

    Abstract: Pairs of text collections are obtained. An individual pair comprises (a) a source text collection which includes a first group of text sequences and (b) an annotated analysis result of the source text collection, comprising a second group of text sequences and a set of evidence mappings generated by an evidence mapping model. An evidence mapping indicates, for a particular text sequence of the second group, another text sequence of the first group which provides evidence for the particular text sequence. A quality metric of the model is obtained using an automated evaluation methodology in which a question is generated from the particular text sequence, and an analysis of a pair of answers (including 10 an answer generated using an evidence mapping) to the question is performed. The quality metric is provided via a programmatic interface.

    DOMAIN SPECIALTY INSTRUCTION GENERATION FOR TEXT ANALYSIS TASKS

    公开(公告)号:US20250029603A1

    公开(公告)日:2025-01-23

    申请号:US18356116

    申请日:2023-07-20

    Abstract: Domain specialty instructions may be generated for performing text analysis tasks. An input text may be received for performing a text analysis task. A domain specialty may be identified for the input text. Specialty domain identifiers may be inserted as part of generating instructions to perform the text analysis task using a pre-trained large language model fine-tuned to a domain that includes multiple domain specialties. The pre-trained large language model may perform the text analysis task on the input text using the generated instructions. A result of the text analysis tsk performed on the input text may be provided.

    LARGE LANGUAGE MODELS PROVIDING EVIDENCE MAPPINGS FOR GENERATED OUTPUT

    公开(公告)号:US20250005298A1

    公开(公告)日:2025-01-02

    申请号:US18344742

    申请日:2023-06-29

    Abstract: Pairs of text collections are obtained. An individual pair comprises (a) a source text collection which includes a first group of text sequences and (b) an annotated analysis result of the source text collection, comprising a second group of text sequences and a set of evidence mappings generated by an evidence mapping model. An evidence mapping indicates, for a particular text sequence of the second group, another text sequence of the first group which provides evidence for the particular text sequence. A quality metric of the model is obtained using an automated evaluation methodology in which a question is generated from the particular text sequence, and an analysis of a pair of answers (including an answer generated using an evidence mapping) to the question is performed. The quality metric is provided via a programmatic interface.

    MEDICAL CONVERSATION SUMMARIZATION STYLE INTELLIGENCE

    公开(公告)号:US20240428002A1

    公开(公告)日:2024-12-26

    申请号:US18339749

    申请日:2023-06-22

    Abstract: A medical audio summarization service receives a medical conversation and an indication of a user preferred summarization style selected from a plurality of available summarization styles to generate a medical summary that conforms to the user preferred summarization style. A transcript is generated via a medical audio transcription service, and the transcript is used by a natural language processing engine (including a large language model) to generate the medical summary. The large language model is trained to be used to generate medical summaries that conform to respective ones of a plurality of user preferred summarization styles. The large language model is trained using training data comprising previously generated summaries and summary interaction metadata generated from user edits and/or feedback.

Patent Agency Ranking