Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating rare pose data. One of the methods includes obtaining a three-dimensional model of a dynamic object, wherein the dynamic object has multiple movable elements that define a plurality of poses of the dynamic object. A plurality of template poses of the dynamic object are used to generate additional poses for the dynamic object including varying angles of one or more key joints of the dynamic object according to the three-dimensional model. Point cloud data is generated for the additional poses generated for the dynamic object.