A freeform varifocal optical assembly includes at least three optical modules including a first plurality of polarization sensitive optical elements including Pancharatnam-Berry phase (PBP) lenses, polarization sensitive hologram (PSH) lenses, metamaterials, or combinations thereof. The plurality of polarization sensitive optical elements of each optical module include a property associated with a Zernike polynomial. Each of the first and the three optical modules are configurable between a plurality of optical powers. The freeform varifocal optical assembly is configurable to output a predetermined wavefront in response to an input wavefront.