GeSeNet: A General Semantic-guided Network with Couple Mask Ensemble for Medical Image Fusion

Jiawei Li, Jinyuan Liu, Shihua Zhou, Qiang Zhang, Nikola Kasabov

Research output: Contribution to journalArticlepeer-review

11 Downloads (Pure)


At present, multimodal medical image fusion technology has become an essential means for researchers and doctors to predict diseases and study pathology. Nevertheless, how to reserve more unique features from different modal source images on the premise of ensuring time efficiency is a tricky problem. To handle this issue, we propose a flexible semantic-guided architecture with a mask-optimized framework in an end-to-end manner, termed as GeSeNet. Specifically, a region mask module is devised to deepen the learning of important information while pruning redundant computation for reducing the runtime. An edge enhancement module and a global refinement module are presented to modify the extracted features for boosting the edge textures and adjusting overall visual performance. In addition, we introduce a semantic module that is cascaded with the proposed fusion network to deliver semantic information into our generated results. Sufficient qualitative and quantitative comparative experiments (i.e., MRI-CT, MRI-PET, and MRI-SPECT) are deployed between our proposed method and ten state-of-the-art methods, which shows our generated images lead the way. Moreover, we also conduct operational efficiency comparisons and ablation experiments to prove that our proposed method can perform excellently in the field of multimodal medical image fusion. The code is available at

Original languageEnglish
Article numberTNNLS-2022-P-25155.R1
Pages (from-to)1-14
Number of pages14
JournalIEEE Transactions on Neural Networks and Learning Systems
Early online date21 Jul 2023
Publication statusPublished online - 21 Jul 2023

Bibliographical note

Publisher Copyright:


  • Image fusion
  • semantic information
  • region mask
  • multimodal medical image
  • Semantics
  • Medical diagnostic imaging
  • Feature extraction
  • Image edge detection
  • Magnetic resonance imaging
  • Discrete wavelet transforms


Dive into the research topics of 'GeSeNet: A General Semantic-guided Network with Couple Mask Ensemble for Medical Image Fusion'. Together they form a unique fingerprint.

Cite this