Provided are an apparatus and method for a non-power-of-2 size cache in a first level memory device to cache data present in a second level memory device having a 2n cache size. A request is to a target address having n bits directed to the second level memory device. A determination is made whether a target index, comprising m bits of the n bits of the target address, is within an index set of the first level memory device. A determination is made of a modified target index in the index set of the first level memory device having at least one index bit that differs from a corresponding at least one index bit in the target index. The request is processed with respect to data in a cache line at the modified target index in the first level memory device.