Systems and methods provide vision-based image compression. In one implementation, inpainting is the vision-based technique selected to augment a conventional signal-processing-based technique. For some regions of a source image, an exemplary system efficiently extracts and organizes structural edge information instead of compressing the regions. In one implementation, the system applies binary curve fitting to capture the edge information. A structure-aware inpainter in the decoder can then restore the regions via the edge information, which occupies very little data space or minimal bandwidth in a bitstream that is transmitted from encoder to decoder. Key visual components of the image can still be conventionally compressed. Extracting edge information for some regions instead of compressing them considerably increases overall image compression.