Systems and methods for identifying body joint location includes obtaining skeletal data, depth data and red, green, and blue (RGB) data pertaining to a user, obtaining, using input data, an estimate of body joint locations (BJLs) and body segment lengths (BSLs), iteratively identifying, based on the depth data and RGB data, probable correct BJLs in a bounded neighborhood around BJLs that are previously obtained, comparing a body segment length associated with the probable correct BJLs and a reference length, identifying candidate BJLs based on comparison, determining a physical orientation of each body segment by segmenting three dimensional (3D) coordinates of each body segment based on the depth data and performing an analysis on each segmented 3D coordinate. A corrected BJL is identified based on a minimal deviation in direction from the physical orientation of a corresponding body segment along with a feature descriptor of the RGB data and depth data.