公开(公告)号：US20210390269A1

公开(公告)日：2021-12-16

申请号：US16900481

申请日：2020-06-12

申请人： Mehdi REZAGHOLIZADEH ,  Vahid PARTOVI NIA ,  Md Akmal HAIDAR ,  Pascal POUPART

发明人： Mehdi REZAGHOLIZADEH ,  Vahid PARTOVI NIA ,  Md Akmal HAIDAR ,  Pascal POUPART

IPC分类号： G06F40/58 ,  G06N3/04 ,  G06N3/08

摘要： A method and machine translation system for bi-directional translation of textual sequences between a first language and a second language are described. The machine translation system includes a first autoencoder configured to receive a vector representation of a first textual sequence in the first language and encode the vector representation of the first textual sequence into a first sentence embedding. The machine translation system also includes a sum-product network (SPN) configured to receive the first sentence embedding and generate a second sentence embedding by maximizing a first conditional probability of the second sentence embedding given the first sentence embedding and a second autoencoder receiving the second sentence embedding, the second autoencoder being trained to decode the second sentence embedding into a vector representation of a second textual sequence in the second language.

公开(公告)号：US20220335303A1

公开(公告)日：2022-10-20

申请号：US17233323

申请日：2021-04-16

申请人： Md Akmal HAIDAR ,  Mehdi REZAGHOLIZADEH

发明人： Md Akmal HAIDAR ,  Mehdi REZAGHOLIZADEH

IPC分类号： G06N3/08 ,  G06N3/04

摘要： Methods, devices and processor-readable media for knowledge distillation using intermediate representations are described. A student model is trained using a Dropout-KD approach in which intermediate layer selection is performed efficiently such that the skip, search, and overfitting problems in intermediate layer KD may be solved. Teacher intermediate layers are selected randomly at each training epoch, with the layer order preserved to avoid breaking information flow. Over the course of multiple training epochs, all of the teacher intermediate layers are used for knowledge distillation. A min-max data augmentation method is also described based on the intermediate layer selection of the Dropout-KD training method.

公开(公告)号：US20200097554A1

公开(公告)日：2020-03-26

申请号：US16143128

申请日：2018-09-26

申请人： Mehdi REZAGHOLIZADEH ,  Md Akmal HAIDAR ,  Alan DO-OMRI ,  Ahmad RASHID

发明人： Mehdi REZAGHOLIZADEH ,  Md Akmal HAIDAR ,  Alan DO-OMRI ,  Ahmad RASHID

IPC分类号： G06F17/28 ,  G06N3/08

摘要： In at least one broad aspect, described herein are systems and methods in which a latent representation shared between two languages is built and/or accessed, and then leveraged for the purpose of text generation in both languages. Neural text generation techniques are applied to facilitate text generation, and in particular the generation of sentences (i.e., sequences of words or subwords) in both languages, in at least some embodiments.

公开(公告)号：US20220122590A1

公开(公告)日：2022-04-21

申请号：US17076794

申请日：2020-10-21

申请人： Md Akmal HAIDAR ,  Chao XING

发明人： Md Akmal HAIDAR ,  Chao XING

IPC分类号： G10L15/16 ,  G10L15/06

摘要： Computer implemented method and system for automatic speech recognition. A first speech sequence is processed, using a time reduction operation of an encoder NN, into a second speech sequence that comprises a second set of speech frame feature vectors that each concatenate information from a respective plurality of speech frame feature vectors included in the first set, wherein the second speech sequence includes fewer speech frame feature vectors than the first speech sequence. The second speech sequence is transformed, using a self-attention operation of the encoder NN, into a third speech sequence that comprises a third set of speech frame feature vectors. The third speech sequence is processed, using a probability operation of the encoder NN, to predict a sequence of first labels corresponding to the third set of speech frame feature vectors. The third speech sequence is also processed using a decoder NN to predict a sequence of second labels corresponding to the third set of speech frame feature vectors.