Methods of forming hetero-layers with reduced surface roughness and bulk defect density on non-native surfaces and the devices formed thereby are described. In one embodiment, the method includes providing a substrate having a top surface with a lattice constant and depositing a first layer on the top surface of the substrate. The first layer has a top surface with a lattice constant that is different from the first lattice constant of the top surface of the substrate. The first layer is annealed and polished to form a polished surface. A second layer is then deposited above the polished surface.