DATA PRE-FETCH FOR LARGE LANGUAGE MODEL (LLM) PROCESSING

    公开(公告)号:US20240370699A1

    公开(公告)日:2024-11-07

    申请号:US18764802

    申请日:2024-07-05

    Abstract: Examples described herein relate to a processor to process constant weight values and key value entries associated with a first transformer kernel of a large language model (LLM) neural network and a circuitry. The circuitry is to: during processing of the constant weight values and key value entries associated with the first transformer kernel of the LLM neural network, pre-fetch constant weight values and key value entries associated with a second transformer kernel of the LLM neural network into a buffer.

Patent Agency Ranking