Floating point video coding is described. In particular, a video encoder/decoder and methods for floating point video coding are discussed. Floating point video coding can be used in the video encoder and decoder to support video content, such as HDR content. Specifically, a video codec structure such as H.264/AVC can be modified in order to support floating point video coding. Within any component of a codec system, some operations can be performed at floating point precision whereas other operations can be performed at fixed point precision.