Method, apparatus, and computer readable medium for producing an optimized matrix triangulation algorithm is described. In one example, tile functions are generated for a matrix triangulation problem. Cost data is measured for the tile functions with respect to a target architecture. The cost data is processed to identify optimal composition of tiles for rows in an iteration space of the matrix triangulation problem. The optimal compositions of tiles are processed to identify optimal composition of rows for triangles in the iteration space. A sequence of tile function invocation based on the optimal compositions of tiles and the optimal compositions of rows is generated.