Described herein are systems, methods, and instrumentalities associated with automatically annotating a 3D image dataset. The 3D automatic annotation may be accomplished based on a 2D manual annotation provided by an annotator and by propagating, using a set of machine-learning (ML) based techniques, the 2D manual annotation through sequences of 2D images associated with the 3D image dataset. The automatically annotated 3D image dataset may then be used to annotate other 3D image datasets upon passing a readiness assessment conducted using another set of ML based techniques. The automatic annotation of the images may be performed progressively, e.g., by processing a subset or batch of images at a time, and the ML based techniques may be trained to ensure consistency between a forward propagation and a backward propagation.