公开(公告)号：US08520808B2

公开(公告)日：2013-08-27

申请号：US12575801

申请日：2009-10-08

申请人： Krishna Dayanidhi ,  Keelan Evanini ,  Phillip Hunter ,  Jackson Liscombe ,  Roberto Pieraccini ,  David Suendermann ,  Zor Gorelov

发明人： Krishna Dayanidhi ,  Keelan Evanini ,  Phillip Hunter ,  Jackson Liscombe ,  Roberto Pieraccini ,  David Suendermann ,  Zor Gorelov

IPC分类号： G10L15/01

CPC分类号： G10L15/01

摘要： A single, subjective numerical rating to evaluate the performance of a telephone-based spoken dialog system. This CE rating is provided by expert human listeners who have knowledge of the design of the dialog system. Different human raters can be trained to achieve a satisfactory level of agreement. Furthermore, a classifier trained on ratings by human experts can reproduce the human ratings with the same degree of consistency. More calls can be given a CE rating than would be possible with limited human resources. More information can be provided about individual calls, e.g., to help decide between two disparate ratings by different human experts.

摘要翻译： 一个单一的主观数字评估来评估基于电话的对话系统的性能。 这个CE评级是由具有对话系统设计知识的专家人员提供的。 不同的人员可以接受培训，达到令人满意的协议水平。 此外，由人类专家对评级进行培训的分类器可以以相同程度的一致性再现人类评级。 更多的电话可以给予CE评级，而不是有限的人力资源。 可以提供关于个人呼叫的更多信息，例如帮助由不同的人类专家在两个不同的评级之间作出决定。

公开(公告)号：US20100091954A1

公开(公告)日：2010-04-15

申请号：US12575801

申请日：2009-10-08

申请人： Krishna DAYANIDHI ,  Keelan Evanini ,  Phillip Hunter ,  Jackson Liscombe ,  Roberto Pieraccini ,  David Suendermann ,  Zor Gorelov

发明人： Krishna DAYANIDHI ,  Keelan Evanini ,  Phillip Hunter ,  Jackson Liscombe ,  Roberto Pieraccini ,  David Suendermann ,  Zor Gorelov

IPC分类号： H04M1/64 ,  G10L15/00 ,  G06F15/18

CPC分类号： G10L15/01

摘要： A single, subjective numerical rating to evaluate the performance of a telephone-based spoken dialog system is disclosed. This CE rating is provided by expert human listeners who have knowledge of the design of the dialog system. Different human raters can be trained to achieve a satisfactory level of agreement. Furthermore, a classifier trained on ratings by human experts can reproduce the human ratings with the same degree of consistency. More calls can be given a CE rating than would be possible with limited human resources. More information can be provided about individual calls, e.g., to help decide between two disparate ratings by different human experts.

摘要翻译： 公开了用于评估基于电话的口语对话系统的性能的单一的主观数字评级。 这个CE评级是由具有对话系统设计知识的专家人员提供的。 不同的人员可以接受培训，达到令人满意的协议水平。 此外，由人类专家对评级进行培训的分类器可以以相同程度的一致性再现人类评级。 更多的电话可以给予CE评级，而不是有限的人力资源。 可以提供关于个人呼叫的更多信息，例如帮助由不同的人类专家在两个不同的评级之间作出决定。

公开(公告)号：US20100268536A1

公开(公告)日：2010-10-21

申请号：US12425892

申请日：2009-04-17

申请人： David Suendermann ,  Keelan Evanini ,  Jackson Liscombe ,  Krishna Dayanidhi ,  Roberto Pieraccini

发明人： David Suendermann ,  Keelan Evanini ,  Jackson Liscombe ,  Krishna Dayanidhi ,  Roberto Pieraccini

IPC分类号： G10L15/06 ,  G10L21/00

CPC分类号： G10L15/22 ,  G10L15/063 ,  G10L15/1815 ,  G10L2015/0635

摘要： A method and apparatus for continuously improving the performance of semantic classifiers in the scope of spoken dialog systems are disclosed. Rule-based or statistical classifiers are replaced with better performing rule-based or statistical classifiers and/or certain parameters of existing classifiers are modified. The replacement classifiers or new parameters are trained and tested on a collection of transcriptions and annotations of utterances which are generated manually or in a partially automated fashion. Automated quality assurance leads to more accurate training and testing data, higher classification performance, and feedback into the design of the spoken dialog system by suggesting changes to improve system behavior.

摘要翻译： 公开了一种在口语对话系统的范围内不断提高语义分类器性能的方法和装置。 基于规则或统计分类器被更好的执行规则或统计分类器替换，和/或修改现有分类器的某些参数。 替代分类器或新参数在手动或部分自动化方式生成的话语集合和训练集上进行训练和测试。 自动化的质量保证通过提出改进来改善系统行为，从而导致更准确的培训和测试数据，更高的分类性能以及对口头对话系统设计的反馈。

公开(公告)号：US08543401B2

公开(公告)日：2013-09-24

申请号：US12425892

申请日：2009-04-17

申请人： David Suendermann ,  Keelan Evanini ,  Jackson Liscombe ,  Krishna Dayanidhi ,  Roberto Pieraccini

发明人： David Suendermann ,  Keelan Evanini ,  Jackson Liscombe ,  Krishna Dayanidhi ,  Roberto Pieraccini

IPC分类号： G10L15/06 ,  G10L15/18 ,  G10L21/00 ,  G06F17/28 ,  G06F17/27 ,  G06F17/21 ,  G06F17/00 ,  G06K9/62

CPC分类号： G10L15/22 ,  G10L15/063 ,  G10L15/1815 ,  G10L2015/0635

摘要： A method and apparatus for continuously improving the performance of semantic classifiers in the scope of spoken dialog systems are disclosed. Rule-based or statistical classifiers are replaced with better performing rule-based or statistical classifiers and/or certain parameters of existing classifiers are modified. The replacement classifiers or new parameters are trained and tested on a collection of transcriptions and annotations of utterances which are generated manually or in a partially automated fashion. Automated quality assurance leads to more accurate training and testing data, higher classification performance, and feedback into the design of the spoken dialog system by suggesting changes to improve system behavior.

摘要翻译： 公开了一种在口语对话系统的范围内不断提高语义分类器性能的方法和装置。 基于规则或统计分类器被更好的执行规则或统计分类器替换，和/或修改现有分类器的某些参数。 替代分类器或新参数在手动或部分自动化方式生成的话语集合和训练集上进行训练和测试。 自动化的质量保证通过提出改进来改善系统行为，从而导致更准确的培训和测试数据，更高的分类性能以及对口头对话系统设计的反馈。

公开(公告)号：US08990082B2

公开(公告)日：2015-03-24

申请号：US13428448

申请日：2012-03-23

申请人： Su-Youn Yoon ,  Derrick Higgins ,  Klaus Zechner ,  Shasha Xie ,  Je Hun Jeon ,  Keelan Evanini

发明人： Su-Youn Yoon ,  Derrick Higgins ,  Klaus Zechner ,  Shasha Xie ,  Je Hun Jeon ,  Keelan Evanini

IPC分类号： G10L15/00 ,  G09B19/06 ,  G10L25/60 ,  G10L15/26 ,  G10L25/78 ,  G10L25/90

CPC分类号： G09B19/06 ,  G10L15/005 ,  G10L15/26 ,  G10L25/60 ,  G10L25/78 ,  G10L25/90

摘要： A method for scoring non-native speech includes receiving a speech sample spoken by a non-native speaker and performing automatic speech recognition and metric extraction on the speech sample to generate a transcript of the speech sample and a speech metric associated with the speech sample. The method further includes determining whether the speech sample is scorable or non-scorable based upon the transcript and speech metric, where the determination is based on an audio quality of the speech sample, an amount of speech of the speech sample, a degree to which the speech sample is off-topic, whether the speech sample includes speech from an incorrect language, or whether the speech sample includes plagiarized material. When the sample is determined to be non-scorable, an indication of non-scorability is associated with the speech sample. When the sample is determined to be scorable, the sample is provided to a scoring model for scoring.

摘要翻译： 用于对非本机语音进行评分的方法包括接收由非本地语音讲话者所说出的语音样本，并在语音样本上执行自动语音识别和度量提取，以产生语音样本的抄本和与语音样本相关联的语音度量。 该方法还包括基于抄本和语音度量确定语音样本是可划分的还是不可读的，其中确定是基于语音样本的音频质量，语音样本的语音量， 语音样本是脱离主题的，语音样本是否包含来自不正确语言的语音，或者言语样本是否包括剽窃材料。 当样本被确定为不可扫描时，不合格性的指示与语音样本相关联。 当样品被确定为可扫描时，将样品提供给评分模型进行评分。

公开(公告)号：US20120323573A1

公开(公告)日：2012-12-20

申请号：US13428448

申请日：2012-03-23

申请人： Su-Youn Yoon ,  Derrick Higgins ,  Klaus Zechner ,  Shasha Xie ,  Je Hun Jeon ,  Keelan Evanini

发明人： Su-Youn Yoon ,  Derrick Higgins ,  Klaus Zechner ,  Shasha Xie ,  Je Hun Jeon ,  Keelan Evanini

IPC分类号： G10L15/00

CPC分类号： G09B19/06 ,  G10L15/005 ,  G10L15/26 ,  G10L25/60 ,  G10L25/78 ,  G10L25/90

摘要： A method for scoring non-native speech includes receiving a speech sample spoken by a non-native speaker and performing automatic speech recognition and metric extraction on the speech sample to generate a transcript of the speech sample and a speech metric associated with the speech sample. The method further includes determining whether the speech sample is scorable or non-scorable based upon the transcript and speech metric, where the determination is based on an audio quality of the speech sample, an amount of speech of the speech sample, a degree to which the speech sample is off-topic, whether the speech sample includes speech from an incorrect language, or whether the speech sample includes plagiarized material. When the sample is determined to be non-scorable, an indication of non-scorability is associated with the speech sample. When the sample is determined to be scorable, the sample is provided to a scoring model for scoring.

摘要翻译： 用于对非本机语音进行评分的方法包括接收由非本地语音讲话者所说出的语音样本，并在语音样本上执行自动语音识别和度量提取，以产生语音样本的抄本和与语音样本相关联的语音度量。 该方法还包括基于抄本和语音度量确定语音样本是可划分的还是不可读的，其中确定是基于语音样本的音频质量，语音样本的语音量， 语音样本是脱离主题的，语音样本是否包含来自不正确语言的语音，或者言语样本是否包括剽窃材料。 当样本被确定为不可扫描时，不合格性的指示与语音样本相关联。 当样品被确定为可扫描时，将样品提供给评分模型进行评分。

公开(公告)号：US20120016672A1

公开(公告)日：2012-01-19

申请号：US13182770

申请日：2011-07-14

申请人： Lei Chen ,  Keelan Evanini ,  Xie Sun

发明人： Lei Chen ,  Keelan Evanini ,  Xie Sun

IPC分类号： G10L15/28

CPC分类号： G09B19/04 ,  G09B19/06 ,  G10L15/04 ,  G10L25/15 ,  G10L25/27 ,  G10L25/60

摘要： Computer-implemented systems and methods are provided for assessing non-native speech proficiency. A non-native speech sample is processed to identify a plurality of vowel sound boundaries in the non-native speech sample. Portions of the non-native speech sample are analyzed within the vowel sound boundaries to extract vowel characteristics. The vowel characteristics are used to identify a plurality of vowel space metrics for the non-native speech sample, and the vowel space metrics are used to determine a non-native speech proficiency score for the non-native speech sample.

摘要翻译： 提供计算机实现的系统和方法来评估非本地语言能力。 处理非本地语音样本以识别非本地语音样本中的多个元音声音边界。 在母语声音边界内分析非本地语音样本的一部分，以提取元音特征。 元音特征用于识别非本地语音样本的多个元音空间度量，并且元音空间度量用于确定非本地语音样本的非本地语音能力得分。

公开(公告)号：US09262941B2

公开(公告)日：2016-02-16

申请号：US13182770

申请日：2011-07-14

申请人： Lei Chen ,  Keelan Evanini ,  Xie Sun

发明人： Lei Chen ,  Keelan Evanini ,  Xie Sun

IPC分类号： G10L15/00 ,  G10L19/06 ,  G10L21/00 ,  G09B19/04 ,  G10L25/60 ,  G09B19/06 ,  G10L15/04 ,  G10L25/27 ,  G10L25/15

CPC分类号： G09B19/04 ,  G09B19/06 ,  G10L15/04 ,  G10L25/15 ,  G10L25/27 ,  G10L25/60

摘要： Computer-implemented systems and methods are provided for assessing non-native speech proficiency. A non-native speech sample is processed to identify a plurality of vowel sound boundaries in the non-native speech sample. Portions of the non-native speech sample are analyzed within the vowel sound boundaries to extract vowel characteristics. The vowel characteristics are used to identify a plurality of vowel space metrics for the non-native speech sample, and the vowel space metrics are used to determine a non-native speech proficiency score for the non-native speech sample.

摘要翻译： 提供计算机实现的系统和方法来评估非本地语言能力。 处理非本地语音样本以识别非本地语音样本中的多个元音声音边界。 在母语声音边界内分析非本地语音样本的一部分，以提取元音特征。 元音特征用于识别非本地语音样本的多个元音空间度量，并且元音空间度量用于确定非本地语音样本的非本地语音能力得分。