A peripheral video generation device includes: a video input unit that inputs peripheral video data captured by a plurality of cameras; a video composition unit that composites the peripheral video data to generate a composite video as viewed from a predetermined viewpoint; a three-dimensional shape estimation unit that estimates a three-dimensional shape of a peripheral object based on the peripheral video data; a shielded area estimation unit that uses an estimation result of the three-dimensional shape to estimate a shielded area not visible from the predetermined viewpoint in the composite video; an inference unit that infers a video of the shielded area using deep learning; and a video superimposition unit that superimposes the video inferred by the inference unit on the shielded area in the composite video.