A method and apparatus queries a table in a database where the table includes at least one column declared to be sparse. A binary large object may be used to store the sparse column data. The object includes a column-id and column-value pair for each non-null value. To answer a query with a constraint on a sparse column, the object is searched for one or more column ids to obtain the column values. Rows whose column values match a constraint are returned. In another embodiment, an internal table is used. Each tuple in the internal table has a column id and a value array indexed by an ordinal row number. To answer a query with a constraint on a sparse column, the column value in the internal table is found and matched against the constraint. If the match is successful, the index of the column value in the internal table is returned.