Multi-frame super-resolution for versatile video coding

Toshiya Hori, Remina Yano, Hiroshi Watanabe, Takeshi Chujoh, Tomohiro Ikai, Eiichi Sasaki, Takuya Suzuki, Norio Ito

研究成果: Conference contribution


The latest video coding standard, Versatile Video Coding (VVC), has been finalized in July 2020. In traditional codecs, it is not possible to refer to different resolution pictures in inter-picture prediction except for inter layer cases. However, VVC does support Reference Picture Resampling (RPR), which allows pictures to refer to different resolution pictures so that video resolution can be adaptively switched in sequence. This work's objective is to improve compression efficiency in the case of RPR. In the preliminary experiments, the compression efficiencies of reduced size encoded case where frames are encoded in half size and up-scaled to their original size and original size encoded case are compared. The result shows that most sequences get worse BD-rate when the sequences are encoded/decoded in reduced size. Here, it is noted that up-scaling is done by VVC Test Model, VTM-10.0, which uses a linear interpolation filter. Thus, the frames cannot recover high-frequency components sufficiently and cannot use temporal correlation. Therefore, this paper proposes a multi-frame super-resolution between frames of different resolutions where some frames are encoded in high quality frames in their original size and other frames in half size by using RPR. The proposed method uses the original size frames, which contain a lot of information, to perform super-resolution to the reduced frame. By doing so, it is observed it saves the bitrate by reducing the frame resolution and increases the quality of reduced frames by using multiframe super-resolution, which leads to improving the compression efficiency.

ホスト出版物のタイトルInternational Workshop on Advanced Imaging Technology, IWAIT 2021
編集者Masayuki Nakajima, Jae-Gon Kim, Wen-Nung Lie, Qian Kemao
出版ステータスPublished - 2021
イベント2021 International Workshop on Advanced Imaging Technology, IWAIT 2021 - Kagoshima, Virtual, Japan
継続期間: 2021 1月 52021 1月 6


名前Proceedings of SPIE - The International Society for Optical Engineering


Conference2021 International Workshop on Advanced Imaging Technology, IWAIT 2021
CityKagoshima, Virtual

ASJC Scopus subject areas

  • 電子材料、光学材料、および磁性材料
  • 凝縮系物理学
  • コンピュータ サイエンスの応用
  • 応用数学
  • 電子工学および電気工学


「Multi-frame super-resolution for versatile video coding」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。