Generating vector representations of visual objects is leveraged in a digital medium environment. For instance, a raster-based visual input object is encoded into a global latent code and individual path latent codes for visual components of the raster visual object are extracted from the global latent code. The path latent codes are decoded and used to generate vector representations of the original raster versions of the visual components. The vector representations are rasterized and composited to generate an output object that simulates a visual appearance of the input object.