A memory access system includes a memory, a controller, and a redundancy elimination unit. The memory is a multi-way set associative memory, and the redundancy elimination unit records M record items. Each record item is used to store a tag of a stored data block in one of storage sets. The controller determines a read data block and a target storage set of the read data block and sends a query message to the redundancy elimination unit. The query message carries a set identifier of the target storage set of the read data block and a tag of the read data block. The redundancy elimination unit determines a record item corresponding to the set identifier of the target storage set, matches the tag of the read data block with a tag of a stored data block in the record item corresponding to the target storage set of the read data block.