Fast H.264/AVC DIRECT mode decision based on mode selection and predicted rate-distortion cost

Xiaocong Jin*, Jun Sun, Yiqing Huang, Jia Su, Takeshi Ikenaga

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


Different encoding modes for variable block size are available in the H.264/AVC standard in order to offer better coding quality. However, this also introduces huge computation time due to the exhaustive check for all modes. In this paper, a fast spatial DIRECT mode decision method for profiles supporting B frame encoding (main profile, high profile, etc.) in H.264/AVC is proposed. Statistical analysis on multiple video sequences is carried out, and the strong relationship of mode selection and rate-distortion (RD) cost between the current DIRECT macroblock (MB) and the co-located MBs is observed. With the check of mode condition, predicted RD cost threshold and dynamic parameter update model, the complex mode decision process can be terminated at an early stage even for small QP cases. Simulation results demonstrate the proposed method can achieve much better performance than the original exhaustive ratedistortion optimization (RDO) based mode decision algorithm by reducing up to 56.8% of encoding time for IBPBP picture group and up to 67.8% of encoding time for IBBPBBP picture group while incurring only negligible bit increment and quality degradation.

Original languageEnglish
Pages (from-to)1653-1662
Number of pages10
JournalIEICE Transactions on Information and Systems
Issue number8
Publication statusPublished - 2011 Aug


  • Adaptive
  • Fast mode decision
  • H.264/AVC
  • Spatial DIRECT

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering
  • Artificial Intelligence


Dive into the research topics of 'Fast H.264/AVC DIRECT mode decision based on mode selection and predicted rate-distortion cost'. Together they form a unique fingerprint.

Cite this