Embodiments are generally directed to selective packing of patches for immersive video. An embodiment of a processing system includes one or more processor cores; and a memory to store data for immersive video, the data including a plurality of patches for multiple projection directions. The system is select the patches for packing, the selection of the patches based at least in part on which of the multiple projection directions is associated with each of the patches. The system is to encode the patches into one or more coded pictures according to the selection of the patches.