Embodiments for video processing, including video coding, video decoding and video transcoding are described. One example method includes performing a conversion between a video having one or more video pictures in a video layer and a bitstream of the video according to a rule, wherein the rule specifies that, in a process of sub-bitstream extraction, removal of (i) a video coding layer (VCL) network abstraction layer (NAL) unit, (ii) filler data NAL units associated with the VCL NAL unit, and (iii) filler payload supplemental enhancement information (SEI) messages associated with the VCL NAL unit is performed regardless of an availability of an external means used to replace a parameter set that is removed during the sub-bitstream extraction.