发明公开
- 专利标题: MULTILINGUAL UNSUPERVISED NEURAL MACHINE TRANSLATION WITH DENOISING ADAPTERS
-
申请号: US17931002申请日: 2022-09-09
-
公开(公告)号: US20230214605A1公开(公告)日: 2023-07-06
- 发明人: Alexandre BÉRARD , Laurent BESACIER , Matthias GALLÉ , Ahmet ÜSTÜN
- 申请人: NAVER CORPORATION
- 申请人地址: KR Seongnam-si
- 专利权人: NAVER CORPORATION
- 当前专利权人: NAVER CORPORATION
- 当前专利权人地址: KR Seongnam-si
- 主分类号: G06F40/58
- IPC分类号: G06F40/58 ; G06F40/126 ; G06F40/47 ; G06N3/08
摘要:
Methods and systems for unsupervised training for a neural multilingual sequence-to-sequence (seq2seq) model. Denoising adapters for each of one or more languages is inserted into an encoder and/or a decoder of the seq2seq model. Parameters of the one or more denoising adapters are trained on a language-specific denoising task using monolingual text for each of the one or more languages. Cross-attention weights of the seq2seq model with the trained denoising adapter layers are fine-tuned on a translation task in at least one of the one or more languages with parallel data.
信息查询