A computer vision-based system provides for load distribution estimation and load rating and vibration serviceability assessment of structures. The system integrates evaluates the structural load carrying capacity, the diagnosis and prognosis of performance and safety, and vibration serviceability. Cameras record images of a structure, and regions of interest are monitored in those images for their displacement and velocity as loading varies. Where the displacement determined exceeds a predetermined threshold, or where the acceleration determined exceeds predetermined limits, or where the distribution of displacements of parts of the structure deviates substantially from an estimated displacement distribution, an output indicating potential problems with the structure is output.