An information processing device includes: a foreground state estimating unit configured to estimate a foreground state of an image using an actual image which is an image to be actually observed; and a visible model updating unit configured to update a background visible model which is visibility of the background of an image and a foreground visible model which is visibility of the foreground using an estimation result of the foreground state.