Evaluation framework for intent authoring processes

    公开(公告)号:US11106875B2

    公开(公告)日:2021-08-31

    申请号:US16417459

    申请日:2019-05-20

    Abstract: Evaluating intent authoring processes, by a processor in a computing environment. Results are received of a simulated intent labeling effort of a dataset comprising utterances of interactive dialog sessions between agents and clients for a given product or service. Figures of merits for respective algorithms used to perform the simulated intent labeling effort are computed. Each of the respective algorithms are evaluated according to the computed figures of merits; and one of the respective algorithms is implemented for labeling intents of a remaining corpus of the synthesized dataset according to parameters evaluated in the computed figures of merits.

    Evaluation framework for intent authoring processes

    公开(公告)号:US11144727B2

    公开(公告)日:2021-10-12

    申请号:US16417444

    申请日:2019-05-20

    Abstract: Evaluating intent authoring processes, by a processor in a computing environment. A dataset comprising utterances of interactive dialog sessions between agents and clients for a given product or service is received. A classification of at least a portion of the utterances is performed for a target intent according to at least one of a plurality of recommendation algorithms, where the classification is performed by an automatic driver invoking the recommendation algorithm and simulating a manual confirmation of the algorithm's decision by a user. A classifier trained with the utterances recommended and confirmed by the automatic driver is automatically evaluated according to at least one of the plurality of evaluation criteria. A report tracking the evaluation results is generated.

    Intent authoring using weak supervision and co-training for automated response systems

    公开(公告)号:US11568856B2

    公开(公告)日:2023-01-31

    申请号:US16949232

    申请日:2020-10-21

    Abstract: A combination of propagation operations and learning algorithms is applied, using a selected set of labeled conversational logs retrieved from a subset of a plurality of conversational logs, to a remaining corpus of the plurality of conversational logs to train an automated response system according to an intent associated with each of the conversational logs. The combination of propagation operations and learning algorithms may include defining the labels by a user for the selected set of the subset of the plurality of conversational logs; training a probabilistic classifier using the defined labels of features of the selected set, wherein the probabilistic classifier produces labeling decisions for the subset of conversational logs; weighting the features of the selected set in a model optimization process; and/or training an additional classifier using the weighted features of the selected set and applying the additional classifier to the remaining corpus.

    Class balancing for intent authoring using search

    公开(公告)号:US10977443B2

    公开(公告)日:2021-04-13

    申请号:US16180902

    申请日:2018-11-05

    Abstract: Embodiments provide for class balancing for intent authoring using search via: receiving a positive example of an utterance associated with an intent, building an in-intent pool of utterances from a conversation log using the positive example in a first search query of the conversation log; adding the in-intent pool of utterances as a positive class to a training dataset; applying Boolean operators to negate the positive example to form a complement example; building an out-intent pool of utterances from the conversation log using the complement example in a first search query of the conversation log; and adding the out-intent pool of utterances as a complement class to the training dataset. The training dataset may be balanced to include a predefined ratio of positive and complement examples. The training dataset may be used to train or retrain an intent classifier.

    CONVERSATIONAL AI WITH MULTI-LINGUAL HUMAN CHATLOGS

    公开(公告)号:US20220391600A1

    公开(公告)日:2022-12-08

    申请号:US17303728

    申请日:2021-06-07

    Abstract: A method, computer system, and computer program product for multi-lingual chatlog training are provided. The embodiment may include receiving, by a processor, a plurality of data related to conversational data in multiple languages. The embodiment may also include assigning an intent label to each conversational data. The embodiment may further include assigning a language label to each conversational data. The embodiment may also include paring the plurality of the data related to the conversational data according to the intent label and the language label. The embodiment may further include training a machine learning model using a multi-lingual and multi-intent conversational data pairing. The embodiment may also include training the machine learning model using a single language and multi-intent conversational data paring.

Patent Agency Ranking