A filter cartridge assembly for an ice making appliance having a rectilinear filter cartridge with a plurality of partitions that are positioned within an internal chamber, form multiple sub-chambers, and create a non-linear pathway for the flow of water through the filter cartridge. Filter media positioned in the sub-chambers of the filter cartridge are configured to remove dissolved solids from water travelling through the filter cartridge and used by the appliance to create ice, including clear ice.