中科微点-全球专利检索

  1. Login
  2. Sign Up

Search Results

5 records (took 0.027 seconds)

    DYNAMIC LOW-RANK ESTIMATION FOR TRANSFORMER-BASED LANGUAGE MODELS
    4.
    发明申请
    DYNAMIC LOW-RANK ESTIMATION FOR TRANSFORMER-BASED LANGUAGE MODELS 有权

    公开(公告)号：US20250029005A1

    公开(公告)日：2025-01-23

    申请号：US18669413

    申请日：2024-05-20

    Applicant: Samsung Electronics Co., Ltd.

    Inventor： Ting Hua ,  Xiao Li ,  Shangqian Gao ,  Yen-Chang Hsu ,  Yilin Shen ,  Hongxia Jin

    IPC: G06N20/00 ,  G10L15/18 ,  G10L15/26

    Abstract: A method includes accessing a plurality of weight matrices of a machine learning model. The method also includes, for each weight matrix, decomposing the weight matrix into a U matrix, an S matrix, and a V matrix using singular value decomposition. The S matrix is a diagonal matrix, and a singular group corresponds to each element in the S matrix. The method further includes, for each weight matrix, determining an importance score of each singular group. The importance score of the singular group represents a change in loss if the singular group is removed from the machine learning model. The method also includes, for each weight matrix, ranking the singular groups across the plurality of weight matrices based on the importance scores. In addition, the method includes, for each weight matrix, identifying one or more of the singular groups to prune based on the ranking of the singular groups.

Patent Agency Ranking