Multi-branch Joint Representation Learning Based on Information Fusion Strategy for Cross-view Geo-localization

Fawei Ge, Yunzhou Zhang, Yixiu Liu, Guiyuan Wang, Sonya Coleman, Dermot Kerr, Li Wang

Research output: Contribution to journalArticlepeer-review

19 Downloads (Pure)

Abstract

Cross-view geo-localization refers to recognizing images of the same geographic target obtained from different platforms (such as drone-view, satellite-view, and ground-view). However, cross-view geo-localization is challenging as image capture using different platforms coupled with extreme viewpoint variations can cause significant changes to the visual image content. Existing methods mainly focus on mining the fine-grained features or the contextual information in neighboring areas, but ignore the complete information of the entire image and the association of contextual information of adjacent regions. Therefore, a multibranch joint representation learning network model based on information fusion strategies (IFSs) is proposed to solve this cross-view geo-localization problem. First, we obtained feature information from the image through global information fusion (GIF) branch and local information fusion (LIF) branch to help the network learn the discernable information in the different images. In addition, a local-guided-GIF (LGGIF) branch is introduced to make local information assist global features to enhance the learning of potential information in the images. Second, we introduced different IFSs in each branch to increase the extraction of contextual information through expanding the global receptive field, thus improving the performance of the model. Finally, a series of experiments is carried out on four prevailing benchmark datasets, namely University-1652, SUES-200, CVUAS, and CVACT datasets. The quantitative comparisons from the experiments clearly indicate that the proposed network framework has great performance. For example, compared with some state-of-the-art methods, the quantitative improvements of the R@1 and AP on the University-1652 datasets are 1.91%, 2.18%, and 1.55%, 2.99% in both tasks, respectively.
Original languageEnglish
Article number5909516
Pages (from-to)1-16
Number of pages16
JournalIEEE Transactions on Geoscience and Remote Sensing
Volume62
Early online date19 Mar 2024
DOIs
Publication statusPublished online - 19 Mar 2024

Bibliographical note

Publisher Copyright:
© 1980-2012 IEEE.

Keywords

  • Geo-localization
  • hybrid information fusion strategies (IFSs)
  • joint representation learning
  • multibranch

Fingerprint

Dive into the research topics of 'Multi-branch Joint Representation Learning Based on Information Fusion Strategy for Cross-view Geo-localization'. Together they form a unique fingerprint.

Cite this