- 专利标题: Auto-regressive neural network systems with a soft attention mechanism using support data patches
-
申请号: US16758461申请日: 2018-10-25
-
公开(公告)号: US11966839B2公开(公告)日: 2024-04-23
- 发明人: Aaron Gerard Antonius van den Oord , Yutian Chen , Danilo Jimenez Rezende , Oriol Vinyals , Joao Ferdinando Gomes de Freitas , Scott Ellison Reed
- 申请人: DeepMind Technologies Limited
- 申请人地址: GB London
- 专利权人: DeepMind Technologies Limited
- 当前专利权人: DeepMind Technologies Limited
- 当前专利权人地址: GB London
- 代理机构: Fish & Richardson P.C.
- 国际申请: PCT/EP2018/079241 2018.10.25
- 国际公布: WO2019/081623A 2019.05.02
- 进入国家日期: 2020-04-23
- 主分类号: G06N3/08
- IPC分类号: G06N3/08 ; G06N20/00
摘要:
A system comprising a causal convolutional neural network to autoregressively generate a succession of values of a data item conditioned upon previously generated values of the data item. The system includes support memory for a set of support data patches each of which comprises an encoding of an example data item. A soft attention mechanism attends to one or more patches when generating the current item value. The soft attention mechanism determines a set of scores for the support data patches, for example in the form of a soft attention query vector dependent upon the previously generated values of the data item. The soft attention query vector is used to query the memory. When generating the value of the data item at a current iteration layers of the causal convolutional neural network are conditioned upon the support data patches weighted by the scores.
公开/授权文献
信息查询