DUAL FORMULATION FOR A COMPUTER VISION RETENTION MODEL

    公开(公告)号:US20250111661A1

    公开(公告)日:2025-04-03

    申请号:US18882629

    申请日:2024-09-11

    Abstract: Transformers are neural networks that learn context and thus meaning by tracking relationships in sequential data. The main building block of transformers is self-attention which allows for cross interaction among all input sequence tokens with each other. This scheme effectively captures short-and long-range spatial dependencies and imposes time and space quadratic complexity in terms of the input sequence length, which enables their use with Natural Language Processing (NLP) and computer vision tasks. While the training parallelism of transformers allows for competitive performance, unfortunately the inference is slow and expensive due to the computational complexity. The present disclosure provides a computer vision retention model that is configured for both parallel training and recurrent inference, which can enable competitive performance during training and fast and memory-efficient inferences during deployment.

Patent Agency Ranking