A system and method for monitoring and controlling build quality during electron beam manufacturing of a build part. The system may include at least one electron beam source to direct at least one electron beam onto a plurality of deposited layers of metallic powder to form a melt pool, a detector to detect in real-time backscattered energy ejected from the melt pool and indicative of a defect in the build part and generate a detection signal representative of the defect. A controller receives and analyzes the detection signal and generates a corrective signal for control of at least one of the actuator and the at least one electron beam source to direct the at least one electron beam onto the plurality of deposited layers of metallic powder to sequentially consolidate patterned portions of the plurality of deposited metallic powder layers to adaptively form the three-dimensional build part.