Methods, systems, and apparatus, including medium-encoded computer program products, for computer aided design and manufacture of physical structures using toolpaths generated by reinforcement learning for use with subtractive manufacturing systems and techniques, include: obtaining, in a computer aided design or manufacturing program, a three dimensional model of a manufacturable object; generating toolpaths that are usable by a computer-controlled manufacturing system to manufacture at least a portion of the manufacturable object by providing at least a portion of the three dimensional model to a machine learning algorithm that employs reinforcement learning, wherein the machine learning algorithm includes one or more scoring functions that include rewards that correlate with desired toolpath characteristics comprising toolpath smoothness, toolpath length, and avoiding collision with the three dimensional model; and providing the toolpaths to the computer-controlled manufacturing system to manufacture at least the portion of the manufacturable object.