A spatially-varying sharpening filter and a color registration module compensate for significant lateral color in poorly corrected optics. In one aspect, a color imaging system includes image-forming optics, a sensor array and a processing module. The processing module includes a color registration module and a spatially-varying sharpening filter. The image-forming optics suffers from lateral chromatic aberration. The sensor array captures color pixels of the chromatically aberrated optical image. The spatially-varying sharpening filter sharpens the image (e.g., reduces the blurring caused by lateral color), and the color registration module realigns different color channels of the image.