A hierarchy machine may be configured as a clustering machine that utilizes local feature embedding to organize visual patterns into nodes that each represent one or more visual patterns. These nodes may be arranged as a hierarchy in which a node may have a parent-child relationship with one or more other nodes. The hierarchy machine may implement a node splitting and tree-learning algorithm that includes hard-splitting of nodes and soft-assignment of nodes to perform error-bounded splitting of nodes into clusters. This may enable the hierarchy machine, which may form all or part of a visual pattern recognition system, to perform large-scale visual pattern recognition, such as font recognition or facial recognition, based on a learned error-bounded tree of visual patterns.