Hardware oriented enhanced category determination based on CTU boundary deblocking strength prediction for SAO in HEVC encoder

High efficiency video coding (HEVC) is a video compression standard that outperforms the predecessor H.264/AVC by doubling the compression efficiency. To enhance the coding accuracy, HEVC adopts sample adaptive offset (SAO), which reduces the distortion of reconstructed pixels using classification based non-linear filtering. In the traditional coding tree unit (CTU) grain based VLSI encoder implementation, during the pixel classification stage, SAO cannot use the raw samples in the boundary of the current CTU because these pixels have not been processed by deblocking filter (DF). This paper proposes a hardware-oriented category determination algorithm based on estimating the deblocking strengths on CTU boundaries and selectively adopting the promising samples in these areas during SAO classification. Compared with HEVC test mode (HM11.0), experimental results indicate that the proposed method achieves an average 0.13%, 0.14%, and 0.12% BD-bitrate reduction (equivalent to 0.0055 dB, 0.0058 dB, and 0.0097 dB increases in PSNR) in CTU sizes of 64 × 64, 32 × 32, and 16 × 16, respectively.