-
公开(公告)号：US11854432B1
公开(公告)日：2023-12-26
申请号：US16459005
申请日：2019-07-01
Applicant: Educational Testing Service
Inventor： Aoife Cahill , Martin Chodorow , Michael Flor
CPC classification number: G09B7/02 , G06F16/285 , G06F40/253 , G06F40/30 , G06N20/00 , G06N20/10 , G06N20/20
Abstract: Systems and methods are provided for processing a group of essays to develop a classifier that detects nonsensical computer-generated essays. A data structure associated with a group of essays is accessed, wherein the group of essays includes nonsensical computer-generated essays and good-faith essays. Both the nonsensical computer-generated essays and the good-faith essays are assigned feature values. The distribution of feature values between the nonsensical computer-generated essays and the good-faith essays is measured. A classifier that detects whether an essay is a nonsensical computer-generated essay is developed, wherein the classifier is developed using the distribution of feature values.
-
公开(公告)号：US11748571B1
公开(公告)日：2023-09-05
申请号：US16878708
申请日：2020-05-20
Applicant: Educational Testing Service
Inventor： Goran Glava{hacek over (s)} , Swapna Somasundaran
IPC: G06F40/289 , G06F40/30 , G06N3/096 , G06N20/00 , G06F40/284 , G06F40/253 , G06N3/08
CPC classification number: G06F40/30 , G06F40/253 , G06F40/284 , G06N3/08 , G06N20/00
Abstract: Data is received that encapsulates a document of text. The text is then segmented into a plurality of semantically coherent units using a coherence-aware text segmentation (CATS) machine learning model. Data is then provided that characterizes the segmenting. Related apparatus, systems, techniques and articles are also described.
-
公开(公告)号：US11568757B1
公开(公告)日：2023-01-31
申请号：US16380680
申请日：2019-04-10
Applicant: Educational Testing Service
Inventor： Blair Lehman , Thomas Florek , Debra Pisacreta , Enruo Guo , Srinivasa Pillarisetti
Abstract: Systems and methods are provided for the design and implementation of experiments that facilitate the investigation of process data. The experiments involve recording the completion of a task by participants and then playing back the video of task completion to automatically probe participants about their affective, behavioral, and cognitive experiences. As a result of this system, information about affective, behavioral, and cognitive processes can be more easily investigated by researchers without computer programming knowledge. Corresponding apparatuses, systems, and methods are also discussed.
-
公开(公告)号：US10937444B1
公开(公告)日：2021-03-02
申请号：US16196716
申请日：2018-11-20
Applicant: Educational Testing Service
Inventor： David Suendermann-Oeft , Lei Chen , Jidong Tao , Shabnam Ghaffarzadegan , Yao Qian
Abstract: A system for end-to-end automated scoring is disclosed. The system includes a word embedding layer for converting a plurality of ASR outputs into input tensors; a neural network lexical model encoder receiving the input tensors; a neural network acoustic model encoder implementing AM posterior probability, word duration, mean value of pitch and mean value of intensity based on a plurality of cues; and a linear regression module, for receiving concatenated encoded features from the neural network lexical model encoder and the neural network acoustic model encoder.
-
公开(公告)号：US10796595B1
公开(公告)日：2020-10-06
申请号：US14942473
申请日：2015-11-16
Applicant: Educational Testing Service
Inventor： Yigal Attali
Abstract: Systems and methods for training raters to rate constructed responses to tasks are described herein. In one embodiment, a plurality of trainee raters are selected without regard to their prior experience. The trainee raters are then train in individual training sessions, during which they are asked to rate responses to a task. Each session presents to the trainee rater the task, a rating rubric, and training responses to the task. The training program receives ratings assigned by the trainee rater to the training responses through a graphical user interface. Upon receiving the assigned rating, the training program presents feedback substantially immediately and determines a score for the trainee rater's assigned rating. Thereafter, qualified raters are selected from the plurality of trainee raters based upon their performance during the training sessions as compared with a statistical model. Operational constructed responses are then assigned to rated by the qualified raters.
-
Patent Agency Ranking
公开(公告)号：US10607188B2
公开(公告)日：2020-03-31
申请号：US14667006
申请日：2015-03-24
Applicant: Educational Testing Service
Inventor： Patrick Charles Kyllonen , Lei Chen , Michelle Paulette Martin , Isaac Bejar , Chee Wee Leong , Joanna Gorin , David Michael Williamson
Abstract: Systems and methods described herein utilize supervised machine learning to generate a model for scoring interview responses. The system may access a training response, which in one embodiment is an audiovisual recording of a person responding to an interview question. The training response may have an assigned human-determined score. The system may extract at least one delivery feature and at least one content feature from the audiovisual recording of the training response, and use the extracted features and the human-determined score to train a response scoring model for scoring interview responses. The response scoring model may be configured based on the training to automatically assign scores to audiovisual recordings of interview responses. The scores for interview responses may be used by interviewers to assess candidates.
-
公开(公告)号：US10373047B2
公开(公告)日：2019-08-06
申请号：US14634203
申请日：2015-02-27
Applicant: Educational Testing Service
Inventor： Derrick Higgins , Lei Chen , Michael Heilman , Klaus Zechner , Nitin Madnani
Abstract: Systems and methods are provided for automatically scoring a constructed response. The constructed response is processed to generate a plurality of numerical vectors that is representative of the constructed response. A model is applied to the plurality of numerical vectors. The model includes an input layer configured to receive the plurality of numerical vectors, the input layer being connected to a following layer of the model via a first plurality of connections. Each of the connections has a first weight. An intermediate layer of nodes is configured to receive inputs from an immediately-preceding layer of the model via a second plurality of connections, each of the connections having a second weight. An output layer is connected to the intermediate layer via a third plurality of connections, each of the connections having a third weight. The output layer is configured to generate a score for the constructed response.
-
公开(公告)号：US09576249B2
公开(公告)日：2017-02-21
申请号：US14218309
申请日：2014-03-18
Applicant: Educational Testing Service
Inventor： Nitin Madnani , Jill Burstein
CPC classification number: G06N99/005 , G06F17/2745 , G06Q10/10 , G09B7/00 , G09B7/02
Abstract: In accordance with the teachings described herein, systems and methods are provided for measuring a user's comprehension of subject matter of a text. A summary generated by the user is received, where the summary summarizes the text. The summary is processed to determine a first numerical measure indicative of a similarity between the summary and a reference summary. The summary is processed to determine a second numerical measure indicative of a degree to which a single sentence of the summary summarizes an entirety of the text. The summary is processed to determine a third numerical measure indicative of a degree of copying in the summary of multi-word sequences present in the text. A numerical model is applied to the first numerical measure, the second numerical measure and the third numerical measure to determine a score for the summary indicative of the user's comprehension of the subject matter of the text.
Abstract translation: 根据本文所述的教导，提供了用于测量用户对文本主题的理解的系统和方法。 收到用户生成的摘要，其中摘要总结了文本。 处理摘要以确定指示摘要和参考摘要之间的相似性的第一数字度量。 处理摘要以确定表示该概要的单个句子总结整个文本的程度的第二数字度量。 处理摘要以确定指示文本中存在的多字序列的总结中的复制程度的第三数值测量。 数字模型应用于第一数值测量，第二数值测量和第三数值测量，以确定表示用户对文本主题的理解的摘要的分数。
-
9.发明授权Computer-implemented systems and methods for scoring of spoken responses based on part of speech patterns 有权基于语音模式的计算机实现的系统和方法，用于评分口语响应公开(公告)号：US09514109B2
公开(公告)日：2016-12-06
申请号：US13739278
申请日：2013-01-11
Applicant: Educational Testing Service
Inventor： Su-Youn Yoon , Suma Bhat
Abstract: Systems and methods are provided for scoring a speech sample. Automatic speech recognition is performed on the speech sample using an automatic speech recognition system to generate a transcription of the sample. Words in the transcription are associated with parts of speech, and part of speech sequences are extracted from the parts of speech associations. A grammar metric is generated based on the part of speech sequences, and the speech sample is scored based on the grammar metric.
Abstract translation: 提供系统和方法来评分语音样本。 使用自动语音识别系统对语音样本执行自动语音识别以产生样本的转录。 转录中的词与词性相关，并且部分语音序列是从部分语音关联中提取出来的。 基于语音序列的一部分生成语法度量，并且基于语法度量对语音样本进行评分。
-
10.发明申请Computer-Implemented Systems and Methods for Generating a Supervised Model for Lexical Cohesion Detection 审中-公开计算机实现的系统和方法，用于生成词汇聚合检测的监督模型公开(公告)号：US20160162806A1
公开(公告)日：2016-06-09
申请号：US14957769
申请日：2015-12-03
Applicant: Educational Testing Service
Inventor： Beata Beigman Klebanov , Michael Flor , Daniel Blanchard
Abstract: Systems and methods are provided for a computer-implemented method for identifying pairs of cohesive words within a text. A supervised model is trained to detect cohesive words within a text to be scored. Training the supervised model includes identifying a plurality of pairs of candidate cohesive words in a training essay and an order associated with the pairs of candidate cohesive words based on an order of words in the training essay. The pairs of candidate cohesive words are filtered to form a set of evaluation pairs. The evaluation pairs are provided via a graphical user interface based on the order associated with the pairs of candidate cohesive words. An indication of cohesion or no cohesion is received for the evaluation pairs via the graphical user interface. The supervised model is trained based on the evaluation pairs and the received indications.
Abstract translation: 为计算机实现的方法提供系统和方法，用于识别文本内的粘性词对。 训练一个监督模型，以检测文本内的凝聚词以进行评分。 培训监督模式包括在培训论文中识别多对候选凝聚词，以及基于训练文章中单词的顺序与候选聚合词对相关联的顺序。 对候选聚合词的对进行过滤以形成一组评估对。 评估对通过图形用户界面基于与候选内聚词对相关联的顺序来提供。 通过图形用户界面为评估对收到凝聚力或无凝聚力的指示。 基于评估对和接收到的指示训练监督模型。
-
-
-
-
-
-
-
-
-
Search Results
150
records
(took 0.026 seconds)
