公开(公告)号：US20190384807A1

公开(公告)日：2019-12-19

申请号：US16007632

申请日：2018-06-13

Applicant: Adobe Inc.

Inventor： Franck Dernoncourt ,  Walter Chang ,  Trung Bui ,  Sean Fitzgerald ,  Sasha Spala ,  Kishore Aradhya ,  Carl Dockhorn

IPC: G06F17/24 ,  G06F17/30 ,  G06N3/08 ,  G06N3/04

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed that collect and analyze annotation performance data to generate digital annotations for evaluating and training automatic electronic document annotation models. In particular, in one or more embodiments, the disclosed systems provide electronic documents to annotators based on annotator topic preferences. The disclosed systems then identify digital annotations and annotation performance data such as a time period spent by an annotator in generating digital annotations and annotator responses to digital annotation questions. Furthermore, in one or more embodiments, the disclosed systems utilize the identified digital annotations and the annotation performance data to generate a final set of reliable digital annotations. Additionally, in one or more embodiments, the disclosed systems provide the final set of digital annotations for utilization in training a machine learning model to generate annotations for electronic documents.

公开(公告)号：US11232255B2

公开(公告)日：2022-01-25

申请号：US16007632

申请日：2018-06-13

Applicant: Adobe Inc.

Inventor： Franck Dernoncourt ,  Walter Chang ,  Trung Bui ,  Sean Fitzgerald ,  Sasha Spala ,  Kishore Aradhya ,  Carl Dockhorn

IPC: G06F40/169

Abstract: Systems, methods, and non-transitory computer-readable media are disclosed that collect and analyze annotation performance data to generate digital annotations for evaluating and training automatic electronic document annotation models. In particular, in one or more embodiments, the disclosed systems provide electronic documents to annotators based on annotator topic preferences. The disclosed systems then identify digital annotations and annotation performance data such as a time period spent by an annotator in generating digital annotations and annotator responses to digital annotation questions. Furthermore, in one or more embodiments, the disclosed systems utilize the identified digital annotations and the annotation performance data to generate a final set of reliable digital annotations. Additionally, in one or more embodiments, the disclosed systems provide the final set of digital annotations for utilization in training a machine learning model to generate annotations for electronic documents.