- 专利标题: Systems and methods for treatment of aberrant responses
-
申请号: US14974721申请日: 2015-12-18
-
公开(公告)号: US11049409B1公开(公告)日: 2021-06-29
- 发明人: Mo Zhang , Jing Chen , Andre Alexander Rupp , David Michael Williamson
- 申请人: Educational Testing Service
- 申请人地址: US NJ Princeton
- 专利权人: Educational Testing Service
- 当前专利权人: Educational Testing Service
- 当前专利权人地址: US NJ Princeton
- 代理机构: Jones Day
- 主分类号: G09B7/02
- IPC分类号: G09B7/02 ; G06F40/20
摘要:
Systems and methods are provided for automatically scoring a response and statistically revaluating whether it can be considered as aberrant. In one embodiment, a constructed response is evaluated via a pre-screening stage and a post-hoc screening stage. The pre-screening stage attempts to determine whether the constructed response is aberrant based on a variety of aberration metrics and criteria. If the constructed response is deemed not to be aberrant, then the post-hoc screening stage attempts to predict a discrepancy between what score an automated scoring system would assigned and what score a human rater would assign to the response. If the discrepancy is sufficiently low, then the constructed response may be scored by an automated scoring engine. On the other hand, if the constructed response failed to pass either of the two stages, then a flag may be raised to indicate that additional human review may be needed.
信息查询