Epitaxial and polycrystalline layers of silicon and silicon-germanium alloys are selectively grown on a semiconductor substrate or wafer by forming over the wafer a thin film masking layer of an oxide of an element selected from scandium, yttrium, lanthanum, cerium, praseodymium, neodymium, samarium, europium, gadolinium, terbium, dysprosium, holmium, erbium, thulium, ytterbium, and lutetium; and then growing the epitaxial layer over the wafer at temperatures below 650.degree. C. The epitaxial and polycrystalline layers do not grow on the masking layer. The invention overcomes the problem of forming epitaxial layers at temperatures above 650.degree. C. by providing a lower temperature process.