Machine learning based adaptive instructions for annotation

    公开(公告)号:US10977518B1

    公开(公告)日:2021-04-13

    申请号:US16355617

    申请日:2019-03-15

    Abstract: Techniques for generating and utilizing machine learning based adaptive instructions for annotation are described. An annotation service can use models to identify edge case data elements predicted to elicit differing annotations from annotators, “bad” data elements predicted to be difficult to annotate, and/or “good” data elements predicted to elicit matching or otherwise high-quality annotations from annotators. These sets of data elements can be automatically incorporated into annotation job instructions provided to annotators, resulting in improved overall annotation results via having efficiently and effectively “trained” the annotators how to perform the annotation task.

    Custom labeling workflows in an active learning-based data labeling service

    公开(公告)号:US11481906B1

    公开(公告)日:2022-10-25

    申请号:US16370723

    申请日:2019-03-29

    Abstract: Techniques for active learning-based data labeling are described. An active learning-based data labeling service enables a user to build and manage large, high accuracy datasets for use in various machine learning systems. Machine learning may be used to automate annotation and management of the datasets, increasing efficiency of labeling tasks and reducing the time required to perform labeling. Embodiments utilize active learning techniques to reduce the amount of a dataset that requires manual labeling. As subsets of the dataset are labeled, this label data is used to train a model which can then identify additional objects in the dataset without manual intervention. The process may continue iteratively until the model converges. This enables a dataset to be labeled without requiring each item in the data set to be individually and manually labeled by human labelers.

Patent Agency Ranking