-
公开(公告)号:US11301626B2
公开(公告)日:2022-04-12
申请号:US16679464
申请日:2019-11-11
Applicant: International Business Machines Corporation
Inventor: Panos Karagiannis , Ladislav Kune , Saloni Potdar , Haoyu Wang , Navneet N. Rao
IPC: G06F40/232 , G06N20/00 , G06F40/30 , G06F40/284 , G06F40/211 , G06F40/253
Abstract: Provided is a method, system, and computer program product for context-dependent spellchecking. The method comprises receiving context data to be used in spell checking. The method further comprises receiving a user input. The method further comprises identifying an out-of-vocabulary (OOV) word in the user input. An initial suggestion pool of candidate words is identified based, at least in part, on the context data. The method then comprises using a noisy channel approach to evaluate a probability that one or more of the candidate words of the initial suggestion pool is an intended word and should be used as a candidate for replacement of the OOV word. The method further comprises selecting one or more candidate words for replacement of the OOV word. The method further comprises outputting the one or more candidates.
-
公开(公告)号:US11270080B2
公开(公告)日:2022-03-08
申请号:US16743661
申请日:2020-01-15
Applicant: International Business Machines Corporation
Inventor: Navneet N. Rao , Ming Tan , Haode Qi , Yang Yu , Panos Karagiannis , Saloni Potdar
IPC: G06F40/247 , G06F40/279 , G06F40/30 , G06F40/289 , G06F40/216 , G06F16/9032
Abstract: A mechanism is provided for implementing a bias detection mechanism that mitigates unintended bias in a conversational agent by leveraging conversational agent definitions, a conversational agent chat logs, and user satisfaction statistics. One or more protected attributes are identified within an utterance from the conversational agent chat logs. Using the identified protected attributes, a replacement utterance with a replacement term is generated for at least one of the identified protected attributes in the utterance. A score is generated for the utterance and the replacement utterance using utterance level relative term importance for protected attributes and regular terms in the utterance and the replacement utterance. Utilizing the scoring, a determination is made as to whether unintended bias exists within the utterance. Responsive to unintended bias being detected, an action is implemented that causes a change to a machine learning model used by the conversational agent.
-
公开(公告)号:US20210319182A1
公开(公告)日:2021-10-14
申请号:US16843872
申请日:2020-04-08
Applicant: International Business Machines Corporation
Inventor: Haode Qi , Ming Tan , Yang Yu , Navneet N. Rao , Saloni Potdar , Haoyu Wang
IPC: G06F40/295 , G06F40/284 , H04L12/58
Abstract: A mechanism is provided to implement suggestion of new entity types with discriminative importance analysis. The mechanism obtains a list of predefined intents from a chatbot designer. The mechanism receives an input sentence having a target intent within the list of predefined intents. The mechanism performs intent-specific importance analysis on the input sentence to generate an importance score for each token in the input sentence. The mechanism ranks the tokens in the input sentence by importance score and outputs a token with a highest importance score as a candidate entity type.
-
公开(公告)号:US10977445B2
公开(公告)日:2021-04-13
申请号:US16265618
申请日:2019-02-01
Applicant: International Business Machines Corporation
Inventor: Yang Yu , Ladislav Kunc , Haoyu Wang , Ming Tan , Saloni Potdar
Abstract: A computer-implemented method includes obtaining a training data set including a plurality of training examples. The method includes generating, for each training example, multiple feature vectors corresponding, respectively, to multiple feature types. The method includes applying weighting factors to feature vectors corresponding to a subset of the feature types. The weighting factors are determined based on one or more of: a number of training examples, a number of classes associated with the training data set, an average number of training examples per class, a language of the training data set, a vocabulary size of the training data set, or a commonality of the vocabulary with a public corpus. The method includes concatenating the feature vectors of a particular training example to form an input vector and providing the input vector as training data to a machine-learning intent classification model to train the model to determine intent based on text input.
-
公开(公告)号:US20200251100A1
公开(公告)日:2020-08-06
申请号:US16265740
申请日:2019-02-01
Applicant: International Business Machines Corporation
Inventor: Ming Tan , Haoyu Wang , Ladislav Kunc , Yang Yu , Saloni Potdar
Abstract: A method includes providing input text to a plurality of multi-task learning (MTL) models corresponding to a plurality of domains. Each MTL model is trained to generate an embedding vector based on the input text. The method further includes providing the input text to a domain identifier that is trained to generate a weight vector based on the input text. The weight vector indicates a classification weight for each domain of the plurality of domains. The method further includes scaling each embedding vector based on a corresponding classification weight of the weight vector to generate a plurality of scaled embedding vectors, generating a feature vector based on the plurality of scaled embedding vectors, and providing the feature vector to an intent classifier that is trained to generate, based on the feature vector, an intent classification result associated with the input text.
-
26.
公开(公告)号:US20200250274A1
公开(公告)日:2020-08-06
申请号:US16267951
申请日:2019-02-05
Applicant: International Business Machines Corporation
Inventor: Ming Tan , Ladislav Kunc , Yang Yu , Haoyu Wang , Saloni Potdar
Abstract: An online version of a sentence representation generation module updated by training a first sentence representation generation module using first labeled data of a first corpus. After training the first sentence representation generation module using the first labeled data, a second corpus of second labeled data is obtained. The second corpus is distinct from the first corpus. A subset of the first labeled data is identified based on similarities between the first corpus and the second corpus. A second sentence representation generation module is trained using the second labeled data of the second corpus and the subset of the first labeled data.
-
公开(公告)号:US11966699B2
公开(公告)日:2024-04-23
申请号:US17350116
申请日:2021-06-17
Applicant: International Business Machines Corporation
Inventor: Abhishek Shah , Ladislav Kunc , Haode Qi , Lin Pan , Saloni Potdar
IPC: G06F40/30 , G06F16/33 , G06F16/35 , G06F40/284 , G06N5/04 , G06N20/00 , G10L15/18 , G06F40/263 , G06F40/279 , G06F40/295 , G06F40/53
CPC classification number: G06F40/284 , G06F16/3344 , G06F16/355 , G06N5/04 , G06N20/00 , G10L15/1822 , G06F40/263 , G06F40/279 , G06F40/295 , G06F40/53
Abstract: A system for classifying a language sample intent by receiving a language sample including a set of features, identifying language sample features, determining a tokenization score for the language sample according to the language sample features, eliminating duplicate features according to the tokenization score, determining a term frequency (tf) according to the identified features and the tokenization score, determining an inverse document frequency (idf) according to the identified features and the tokenization score, and generating a term frequency-inverse document frequency (tf-idf) matrix for the identified features.
-
公开(公告)号:US20220358851A1
公开(公告)日:2022-11-10
申请号:US17302550
申请日:2021-05-06
Applicant: International Business Machines Corporation
Inventor: Dakuo Wang , Mo Yu , Chuang Gan , Saloni Potdar
Abstract: In an approach to generating question answer pairs, one or more computer processors receive a corpus of text. One or more computer processors extract one or more key concepts from the corpus of text. Based on the one or more key concepts, one or more computer processors generate one or more questions associated with the key concepts, where the one or more key concepts are answers to the one or more generated questions. One or more computer processors display the one or more generated questions and the answers to the one or more generated questions.
-
公开(公告)号:US11436528B2
公开(公告)日:2022-09-06
申请号:US16543117
申请日:2019-08-16
Applicant: International Business Machines Corporation
Inventor: Haoyu Wang , Ming Tan , Dakuo Wang , Chuang Gan , Saloni Potdar
Abstract: A method includes determining, based on an input data sample, a set of probabilities. Each probability of the set of probabilities is associated with a respective label of a set of labels. A particular probability associated with a particular label indicates an estimated likelihood that the input data sample is associated with the particular label. The method includes modifying the set of probabilities based on a set of adjustment factors to generate a modified set of probabilities. The set of adjustment factors is based on a first relative frequency distribution and a second relative frequency distribution. The first relative frequency distribution indicates for each label of the set of labels, a frequency of occurrence of the label among training data. The second relative frequency distribution indicates for each label of the set of labels, a frequency of occurrence of the label among post-training data provided to the trained classifier.
-
公开(公告)号:US11270077B2
公开(公告)日:2022-03-08
申请号:US16411076
申请日:2019-05-13
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor: Ming Tan , Ladislav Kunc , Yang Yu , Haoyu Wang , Saloni Potdar
Abstract: A computing device receives a natural language input from a user. The computing device routes the natural language input from an active domain node of multiple domain nodes of a multi-domain context-based hierarchy to a leaf node of the domain nodes by selecting a parent domain node in the hierarchy until an off-topic classifier labels the natural language input as in-domain and then selecting a subdomain node in the hierarchy until an in-domain classifier labels the natural language input with a classification label, each of the plurality of domain nodes comprising a respective off-topic classifier and a respective in-domain classifier trained for a respective domain node. The computing device outputs the classification label determined by the leaf node.
-
-
-
-
-
-
-
-
-