摘要:
A hardware data compressor that compresses an input block of characters by replacing strings of characters in the input block with back pointers to matching strings earlier in the input block. A hash table is used in searching for the matching strings in the input block. A plurality of hash index generators each employs a different hashing algorithm on an initial portion of the strings of characters to be replaced to generate a respective index. The hardware data compressor also includes an indication of a type of the input block of characters. A selector selects the index generated by of one of the plurality hash index generators to index into the hash table based on the type of the input block.
摘要:
A system is disclosed comprising a transceiver, transcoder, memory, and a processor for receiving raw data, partitioning the raw data into substrings of predetermined length, assigning each substring to a corresponding predetermined frequency based upon a data set or first lookup table based on the substring's given pattern, and transmitting said frequency using an antenna. Embodiments include a compression component for receiving raw data as input, breaking the raw data into subsets of predetermined length, comparing the raw data to a second lookup table, the second lookup table comprising all possible bit patterns for a file of the length of the raw data, wherein the possible bit patterns are partitioned in n-bit partitions, the n-bit partitions having a corresponding assigned value, the values of which are assembled by a given function so as to produce a code for each possible bit pattern.
摘要:
Memory system operations are extended for a data processor by an application programming interface API, including a set of operations and parameters for the operations, which provides for data compression and decompression during or in conjunction with processes for moving data between memory elements of the memory system. The set of operations can be configured to use the parameters and perform the operations of the API. The API can support moves between memory having a first access latency, such as memory integrated on the same chip as a processor core, and memory having a second access latency that is longer than the first access latency, such as memory on a different integrated circuit than the processor core.
摘要:
Disclosed herein are systems and methods for compressing structured or semi-structured data in a horizontal manner achieving compression ratios similar to vertical compression. Collections include structured or semi-structured data include a number of fields and are described using a schema. Fields include information having semantic similarity and are compressed using methods suitable for compressing the type of data. Data of a collection is compressed after fragmentation or may be normalized prior to compression. Data with semantic similarity is compressed using token tables and/or n-gram tables, where higher weighted, consisting of the product of frequency and length, occurring values may be stored in the lower numbered indices of the data table. Records include record descriptor bytes, field descriptor bytes, zero or more array descriptor bytes, zero or more object descriptor bytes, or bytes representing the data associated with the record. Data is indexed or compressed by a suitable module.
摘要:
Exemplary embodiments provide a data compression technique which chooses a compression method without compressing data. A storage system comprises a storage media and a controller. The controller is operable to: determine a compression method to be used to compress a data block of uncompressed data based on one or more characteristics of data content of the uncompressed data prior to compressing the data block; and compress the data block of the uncompressed data using the determined compression method. In some embodiments, the controller is operable to determine the compression method based on a compression rule which relates one or more characteristics of data content and compression methods. In specific embodiments, the storage system further comprises a flash memory device which includes the controller to determine the compression method and to compress the data block.
摘要:
Methods and systems are disclosed for adaptive compression is disclosed. The compression is adaptive based on one or more criteria related to the system, including but not limited to: the centralized authority, one or more remote devices, the communication channel, and/or the data communicated. For example, the compression may be adaptive based on one or more criteria related to the remote device, including dynamic and/or static criteria of the remote device. In this way, the adaptive compression may be reduce the amount of data transmitted from the remote device to the centralized authority, thereby conserving bandwidth.
摘要:
Exemplary methods, computer systems, and computer program products for processing a previously compressed data stream in a computer environment are provided. In one embodiment, the computer environment is configured for separating a previously compressed data stream into an input data block including a header input block having a previously compressed header. Sequences of bits are included with the input data block. Compression scheme information is derived from the previously compressed header. The input data block is accessed and recompressed following the header input block in the previously compressed data stream one at a time using block-image synchronization information. Access to the block-image synchronization information is initialized by the compression scheme information to generate an output data block. The block-image synchronization information is used to provide decompression information to facilitate decompression of the results of the output data block.
摘要:
Memory system operations are extended for a data processor by an application programming interface API, including a set of operations and parameters for the operations, which provides for data compression and decompression during or in conjunction with processes for moving data between memory elements of the memory system. The set of operations can be configured to use the parameters and perform the operations of the API. The API can support moves between memory having a first access latency, such as memory integrated on the same chip as a processor core, and memory having a second access latency that is longer than the first access latency, such as memory on a different integrated circuit than the processor core.
摘要:
An encryption and authentication technique that achieves enhanced integrity verification through assured error-propagation using a multistage sequence of pseudorandom permutations. The present invention generates intermediate data-dependent cryptographic variables at each stage, which are systematically combined into feedback loops. The encryption technique also generates an authentication tag without any further steps that is N times longer than the block size where N is the number of pseudorandom permutations used in the encipherment of each block. The authentication tag provides a unique mapping to the plaintext for any number of plaintext blocks that is less than or equal to N. In addition to being a stand alone encryption algorithm, the disclosed technique is applicable to any mode that uses pseudorandom permutations such as, key dependent lookup tables, S-Boxes, and block ciphers such as RC5, TEA, and AES.
摘要:
Embodiments of the present invention relate to the database field, and in particular, to a data transmission method and apparatus, so as to reduce overheads for data transmission between nodes in a distributed database and lighten network load. In the embodiments of the present invention, a DN determines a column that has a distribution rule of parameters in the column; the DN determines, according to a data type of the parameters of the column and the distribution rule of the parameters in the column, a compression algorithm corresponding to the column; the DN compresses the column using the compression algorithm; and the DN sends a compressed column to a target node. In this way, the overheads for the data transmission between the nodes are reduced, and the network load is lightened.