System and method for robust evaluation of the user experience in automated spoken dialog systems
    1.
    发明授权
    System and method for robust evaluation of the user experience in automated spoken dialog systems 有权
    自动语音对话系统用户体验的系统和方法

    公开(公告)号:US08520808B2

    公开(公告)日:2013-08-27

    申请号:US12575801

    申请日:2009-10-08

    IPC分类号: G10L15/01

    CPC分类号: G10L15/01

    摘要: A single, subjective numerical rating to evaluate the performance of a telephone-based spoken dialog system. This CE rating is provided by expert human listeners who have knowledge of the design of the dialog system. Different human raters can be trained to achieve a satisfactory level of agreement. Furthermore, a classifier trained on ratings by human experts can reproduce the human ratings with the same degree of consistency. More calls can be given a CE rating than would be possible with limited human resources. More information can be provided about individual calls, e.g., to help decide between two disparate ratings by different human experts.

    摘要翻译: 一个单一的主观数字评估来评估基于电话的对话系统的性能。 这个CE评级是由具有对话系统设计知识的专家人员提供的。 不同的人员可以接受培训,达到令人满意的协议水平。 此外,由人类专家对评级进行培训的分类器可以以相同程度的一致性再现人类评级。 更多的电话可以给予CE评级,而不是有限的人力资源。 可以提供关于个人呼叫的更多信息,例如帮助由不同的人类专家在两个不同的评级之间作出决定。

    SYSTEM AND METHOD FOR ROBUST EVALUATION OF THE USER EXPERIENCE IN AUTOMATED SPOKEN DIALOG SYSTEMS
    2.
    发明申请
    SYSTEM AND METHOD FOR ROBUST EVALUATION OF THE USER EXPERIENCE IN AUTOMATED SPOKEN DIALOG SYSTEMS 有权
    用于在自动对讲机系统中用户体验的鲁棒评估的系统和方法

    公开(公告)号:US20100091954A1

    公开(公告)日:2010-04-15

    申请号:US12575801

    申请日:2009-10-08

    IPC分类号: H04M1/64 G10L15/00 G06F15/18

    CPC分类号: G10L15/01

    摘要: A single, subjective numerical rating to evaluate the performance of a telephone-based spoken dialog system is disclosed. This CE rating is provided by expert human listeners who have knowledge of the design of the dialog system. Different human raters can be trained to achieve a satisfactory level of agreement. Furthermore, a classifier trained on ratings by human experts can reproduce the human ratings with the same degree of consistency. More calls can be given a CE rating than would be possible with limited human resources. More information can be provided about individual calls, e.g., to help decide between two disparate ratings by different human experts.

    摘要翻译: 公开了用于评估基于电话的口语对话系统的性能的单一的主观数字评级。 这个CE评级是由具有对话系统设计知识的专家人员提供的。 不同的人员可以接受培训,达到令人满意的协议水平。 此外,由人类专家对评级进行培训的分类器可以以相同程度的一致性再现人类评级。 更多的电话可以给予CE评级,而不是有限的人力资源。 可以提供关于个人呼叫的更多信息,例如帮助由不同的人类专家在两个不同的评级之间作出决定。

    SYSTEM AND METHOD FOR IMPROVING PERFORMANCE OF SEMANTIC CLASSIFIERS IN SPOKEN DIALOG SYSTEMS
    4.
    发明申请
    SYSTEM AND METHOD FOR IMPROVING PERFORMANCE OF SEMANTIC CLASSIFIERS IN SPOKEN DIALOG SYSTEMS 有权
    用于提高语音对话系统中语义分类器性能的系统和方法

    公开(公告)号:US20100268536A1

    公开(公告)日:2010-10-21

    申请号:US12425892

    申请日:2009-04-17

    IPC分类号: G10L15/06 G10L21/00

    摘要: A method and apparatus for continuously improving the performance of semantic classifiers in the scope of spoken dialog systems are disclosed. Rule-based or statistical classifiers are replaced with better performing rule-based or statistical classifiers and/or certain parameters of existing classifiers are modified. The replacement classifiers or new parameters are trained and tested on a collection of transcriptions and annotations of utterances which are generated manually or in a partially automated fashion. Automated quality assurance leads to more accurate training and testing data, higher classification performance, and feedback into the design of the spoken dialog system by suggesting changes to improve system behavior.

    摘要翻译: 公开了一种在口语对话系统的范围内不断提高语义分类器性能的方法和装置。 基于规则或统计分类器被更好的执行规则或统计分类器替换,和/或修改现有分类器的某些参数。 替代分类器或新参数在手动或部分自动化方式生成的话语集合和训练集上进行训练和测试。 自动化的质量保证通过提出改进来改善系统行为,从而导致更准确的培训和测试数据,更高的分类性能以及对口头对话系统设计的反馈。

    System and method for building optimal state-dependent statistical utterance classifiers in spoken dialog systems
    7.
    发明授权
    System and method for building optimal state-dependent statistical utterance classifiers in spoken dialog systems 有权
    在口语对话系统中建立最优状态依赖统计话语分类器的系统和方法

    公开(公告)号:US08682669B2

    公开(公告)日:2014-03-25

    申请号:US12545718

    申请日:2009-08-21

    摘要: A system and a method to generate statistical utterance classifiers optimized for the individual states of a spoken dialog system is disclosed. The system and method make use of large databases of transcribed and annotated utterances from calls collected in a dialog system in production and log data reporting the association between the state of the system at the moment when the utterances were recorded and the utterance. From the system state, being a vector of multiple system variables, subsets of these variables, certain variable ranges, quantized variable values, etc. can be extracted to produce a multitude of distinct utterance subsets matching every possible system state. For each of these subset and variable combinations, statistical classifiers can be trained, tuned, and tested, and the classifiers can be stored together with the performance results and the state subset and variable combination. Once the set of classifiers and stored results have been put into a production system, for a given system state, the classifiers resulting in optimum performance can be selected from the result list and used to perform utterance classification.

    摘要翻译: 公开了一种用于生成针对口语对话系统的各个状态而优化的统计话语分类器的系统和方法。 系统和方法利用生产对话系统中收集的呼叫的转录和注释语句的大型数据库,并且记录在记录语音时的系统状态与语音之间的关联。 从系统状态,作为多个系统变量的向量,可以提取这些变量的子集,某些可变范围,量化变量值等,以产生与每个可能的系统状态匹配的众多不同的话语子集。 对于这些子集和变量组合中的每一个,可以对统计分类器进行训练,调整和测试,并将分类器与性能结果以及状态子集和变量组合一起存储。 一旦将分类器和存储结果集合放入生产系统中,对于给定的系统状态,可以从结果列表中选择导致最佳性能的分类器,并用于执行话语分类。

    SYSTEM AND METHOD FOR BUILDING OPTIMAL STATE-DEPENDENT STATISTICAL UTTERANCE CLASSIFIERS IN SPOKEN DIALOG SYSTEMS
    8.
    发明申请
    SYSTEM AND METHOD FOR BUILDING OPTIMAL STATE-DEPENDENT STATISTICAL UTTERANCE CLASSIFIERS IN SPOKEN DIALOG SYSTEMS 有权
    系统和方法在SPOKEN对话系统中构建最佳状态依赖统计学分类器

    公开(公告)号:US20110046951A1

    公开(公告)日:2011-02-24

    申请号:US12545718

    申请日:2009-08-21

    IPC分类号: G10L15/08

    摘要: A system and a method to generate statistical utterance classifiers optimized for the individual states of a spoken dialog system is disclosed. The system and method make use of large databases of transcribed and annotated utterances from calls collected in a dialog system in production and log data reporting the association between the state of the system at the moment when the utterances were recorded and the utterance. From the system state, being a vector of multiple system variables, subsets of these variables, certain variable ranges, quantized variable values, etc. can be extracted to produce a multitude of distinct utterance subsets matching every possible system state. For each of these subset and variable combinations, statistical classifiers can be trained, tuned, and tested, and the classifiers can be stored together with the performance results and the state subset and variable combination. Once the set of classifiers and stored results have been put into a production system, for a given system state, the classifiers resulting in optimum performance can be selected from the result list and used to perform utterance classification.

    摘要翻译: 公开了一种用于生成针对口语对话系统的各个状态而优化的统计话语分类器的系统和方法。 系统和方法利用生产对话系统中收集的呼叫的转录和注释语句的大型数据库,并记录在记录语音时的系统状态与语音之间的关联。 从系统状态,作为多个系统变量的向量,可以提取这些变量的子集,某些可变范围,量化变量值等,以产生与每个可能的系统状态匹配的众多不同的话语子集。 对于这些子集和变量组合中的每一个,可以对统计分类器进行训练,调整和测试,并将分类器与性能结果以及状态子集和变量组合一起存储。 一旦将分类器和存储结果集合放入生产系统中,对于给定的系统状态,可以从结果列表中选择导致最佳性能的分类器,并用于执行话语分类。