A method and device for compressing and storing data based on a sparse matrix are disclosed. According to the method, attribute data and value data of a first data are respectively stored, that is, the attribute data of the first data and an attribute identifiers corresponding to the attribute data are stored in the first storage file, and the attribute identifiers corresponding to the attribute data are set as dimension data for determining the first data. Thus second storage file only stores the dimension data and the single value data. In conventional art, five storage domains are used to store the sparse matrix, where real data value is stored in a data value domain. Since many of the data values of each node are the same, and the same data are stored repeatedly, thus storage space is wasted.