Semi-supervised attention based merging network with hybrid dilated convolution module for few-shot HDR video reconstruction

Fengshan Zhao*, Qin Liu, Takeshi Ikenaga

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Deep learning based methods for high dynamic range (HDR) video reconstruction require collecting large-scale HDR video dataset with ground truth which is time-consuming. Recent training strategies under the few-shot learning paradigm, which aim to build an effective model upon only a few labeled samples, have shown success in image classification and image segmentation. In this paper, a semi-supervised learning based framework for few-shot HDR video reconstruction is proposed. An attention based merging network with the hybrid dilated convolution module is used to recover missing contents and remove artifacts. The hybrid dilated convolution module extracts additional features from ill-exposed regions and the attention module corrects them to suppress harmful information. In the semi-supervised framework, designed training losses for the supervised branch and the unsupervised branch are utilized to constrain the network during training under the few-shot scenario. Experimental results show that the proposed method trained with only 5 labeled samples and 45 unlabeled samples achieves a PSNR score of 41.664dB on synthetic evaluation dataset, compared with 35.201dB which is the best score among supervised methods trained in the same few-shot condition.

Original languageEnglish
Pages (from-to)37409-37430
Number of pages22
JournalMultimedia Tools and Applications
Volume83
Issue number13
DOIs
Publication statusPublished - 2024 Apr

Keywords

  • Computational photography
  • Few-shot learning
  • High dynamic range (HDR) video reconstruction
  • Semi-supervised learning

ASJC Scopus subject areas

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Semi-supervised attention based merging network with hybrid dilated convolution module for few-shot HDR video reconstruction'. Together they form a unique fingerprint.

Cite this