A coated article is provided so as to include a solar control coating having an infrared (IR) reflecting layer and a breaker layer, one or both of which may be of or include a material such as NiCr, NiCrNx, NbCr, NbCrNx, NbZr, NbZrNx, Nb and/or NbNx. Each of the IR reflecting layer and the breaker layer is sandwiched between at least a pair of dielectric layers of a material such as silicon nitride or the like. In certain example embodiments of this invention, the IR reflecting layer is substantially thicker than the breaker layer. Such coated articles may be used in the context of windows such as monolithic or IG windows, and may optionally be heat treated (e.g., thermally tempered) in certain instances and may be substantially thermally stable.