A device implementing an adaptive assembly guidance system includes an image sensor and a processor configured to capture, using the image sensor, an image of a set of connectable components. The processor is further configured to process the captured image to detect individual connectable components of the set of connectable components and to detect a current configuration of the set of connectable components. The processor is further configured to determine, based at least in part on the detected individual connectable components of the set of connectable components, a recommended configuration of the set of connectable components. The processor is further configured to display information for assembling the set of connectable components into the recommended configuration from the current configuration.