Provided are an image processing device, an imaging element, an image processing method, and an image processing program that satisfactorily generate a plurality of polarized image data from polarized image data acquired from an imaging element. A processor (200B) of an image processing device (200) performs an acquisition process of acquiring first image data from an imaging element (100) in which four first-polarizers having different polarization directions are regularly provided on pixels arranged in a two-dimensional manner, a first polarized image data generation process of performing a demosaicing process on the first image data to generate four pieces of first polarized image data having different polarization directions, and a second polarized image data generation process of generating four or less pieces of second polarized image data by using the four pieces of first polarized image data and a relationship between the polarization directions of the four first-polarizers stored in the memory (200C).