Systems and methods described herein provide techniques for determining a visual frailty score by processing video data of a subject. Various features maybe used to determine the visual frailty score, including but not limited to, spinal mobility features, gait measurements, behavior features, and body composition data. The various features may be extracted from the video data using different techniques. The various features may be processed using one or more machine learning models to determine the visual frailty score.