A reflector (e.g., mirror) for use in a solar collector or the like is provided. In certain example embodiments of this invention, a reflector is made by (a) forming a reflective coating on a thin substantially flat glass substrate (the thin glass substrate may or may not be pre-bent prior to the coating being applied thereto), (b) optionally, if the glass substrate in (a) was not prebent, then cold-bending the glass substrate with the reflective coating thereon; and (c) applying a plate or frame member to the thin bent glass substrate with the coating thereon from (a) and/or (b), the plate or frame member (which may be another thicker pre-bent glass sheet, for example) for maintaining the thin glass substrate and coating thereon in a desired bent orientation in a final product which may be used as parabolic trough or dish type reflector in a concentrating solar power apparatus or the like.