A method and apparatus for video processing of reconstructed video in a reconstruction loop are disclosed. An embodiment according to the present invention performs padding for the blocks in the slice boundaries according to a processing order. Each of the one or more reconstructed blocks at slice boundaries is padded according to a padding order. In-loop filtering is applied to the reconstructed slice according to the processing order, wherein the reconstructed slice is filtered by referencing the one or more padded blocks. The padding order can be vertical-edge first and then horizontal-edge, or horizontal-edge first and then vertical-edge. Various embodiments are disclosed regarding whether padding for a later padded block in the processing order is skipped or not in the overlapped area. The present invention is also applied to adaptation blocks for block-based in-loop filtering in slice boundaries.