A high solid polyester polyol/silica nanocomposite resin with low viscosity is synthesized by an in situ polymerization method, namely conducting condensation polymerization of polyol/diacid monomer with a molar ratio of 1.1˜1.5 in the presence of colloidal silica nanoparticles with a diameter of 10˜150 nm. The resulted nanocomposite resins has a solid content of >95%, molecular weight of 1000˜5000 g/mol and hydroxyl value of 80-200 mg KOH/g, and a viscosity in the range of 800-2000 mPa.s. The viscosities are especially lower than those obtained by a blending method. The nanosilica content in the nanocomposite resin is 1˜15% (based on the total solid content of nanocomposite resin). The high solid polyester polyol/silica nanocomposite resin was curable with isocyanate or melamine to prepare high solid transparent nanocomposite coatings with obviously improved abrasion and scratch resistance, hardness, weatherability etc.