A color stabilized composite material combines polymers with cellulosic natural fibers or wood or wood based composite materials to manufacture an array of products for both indoor and outdoor applications. The invention provides the means to enhance the color stability of such products from the negative effects of ultraviolet radiation and weather in addition to improving the physical properties of the overall fiber-polymer matrix. In one aspect, the invention provides a composite material including a natural fiber, such as wood flour, and a polymer resin, wherein the natural fiber has been pretreated with a bleaching or oxidizing agent. In another aspect, the invention provides a color stabilized composite building material including an elongated substrate, and a thermoplastic material coating disposed on a longitudinal circumference of the substrate, which may be wood or a wood-based composite material such as plywood, cellulosic fiberboard, particle board, waferboard, flakeboard, chipboard, or oriented strand board.