Application of Multi-modal Fusion Attention Mechanism in Semantic Segmentation

Yunlong Liu*, Osamu Yoshie, Hiroshi Watanabe

*この研究の対応する著者

研究成果: Conference contribution

抄録

The difficulty of semantic segmentation in computer vision has been reintroduced as a topic of interest for researchers thanks to the advancement of deep learning algorithms. This research aims into the logic of multi-modal semantic segmentation on images with two different modalities of RGB and Depth, which employs RGB-D images as input. For cross-modal calibration and fusion, this research presents a novel FFCA Module. It can achieve the goal of enhancing segmentation results by acquiring complementing information from several modalities. This module is plug-and-play compatible and can be used with existing neural networks. A multi-modal semantic segmentation network named FFCANet has been designed to test the validity, with a dual-branch encoder structure and a global context module developed using the classic combination of ResNet and DeepLabV3+ backbone. Compared with the baseline, the model used in this research has drastically improved the accuracy of the semantic segmentation task.

本文言語English
ホスト出版物のタイトルComputer Vision – ACCV 2022 - 16th Asian Conference on Computer Vision, Proceedings
編集者Lei Wang, Juergen Gall, Tat-Jun Chin, Imari Sato, Rama Chellappa
出版社Springer Science and Business Media Deutschland GmbH
ページ378-397
ページ数20
ISBN(印刷版)9783031262920
DOI
出版ステータスPublished - 2023
イベント16th Asian Conference on Computer Vision, ACCV 2022 - Macao, China
継続期間: 2022 12月 42022 12月 8

出版物シリーズ

名前Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
13847 LNCS
ISSN(印刷版)0302-9743
ISSN(電子版)1611-3349

Conference

Conference16th Asian Conference on Computer Vision, ACCV 2022
国/地域China
CityMacao
Period22/12/422/12/8

ASJC Scopus subject areas

  • 理論的コンピュータサイエンス
  • コンピュータサイエンス一般

フィンガープリント

「Application of Multi-modal Fusion Attention Mechanism in Semantic Segmentation」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル