Column based defect management techniques are presented. Each column of the memory has an associated isolation latch or register whose value indicates whether the column is defective, but in addition to this information, for columns marked as defective, additional information is used to indicate whether the column as a whole is to be treated as defective, or whether just individual bits of the column are defective. The defective elements can then be re-mapped to a redundant element at either the appropriate bit or column level based on the data. When a column is bad, but only on the bit level, the good bits can still be used for data, although this may be done at a penalty of under programming for some bits, as is described further below. A self contained Built In Self Test (BIST) flow constructed to collect the bit information through a set of column tests is also described. Based on this information, the bad bits can be extracted and re-grouped into bytes by the controller or on the memory to more efficiently use the column redundancy area.