Integration of text and image analysis for flood event image recognition

Min Jing, Scotney Bryan, Sonya Coleman, T.Martin McGinnity, Xiubo Zhang, Stephen Kelly, Khurshid Ahmad, Antje Schlaf, Sabine Gr¨under-Fahrer, Gerhard Heyer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)
88 Downloads (Pure)

Abstract

Flood event monitoring plays an important role for emergency management. With the fast growth of social media, a large number of images and videos are uploaded and searched on the internet during disasters, which can be used as “sensors” for improving efficiency of emergency management. This work proposes a novel framework in which the rich information available from social media is incorporated with image analysis to enhance image retrieval for disaster management. The text associated with images of flooding events was used to extract prominent words associated with flooding. The image features are represented by a histogram of visual words obtained using the Bag-of-Words (BoW) model. The text and image analysis are integrated at the feature level, in which the text features are conjoined directly with image features. The proposed approach was evaluated based on two flood event corpuses obtained from the US Federal Emergency Management Agency media library and public Facebook pages and groups related to flood and flood aid (in German). The experimental results demonstrate the improved performance of image recognition after incorporating the text features, which suggests the potential to enhance the efficiency of emergency management.
Original languageEnglish
Title of host publicationUnknown Host Publication
PublisherIEEE
Number of pages6
ISBN (Print)978-1-5090-3409-3
DOIs
Publication statusAccepted/In press - 21 Jun 2016
EventSignals and Systems Conference (ISSC), 2016 27th Irish -
Duration: 21 Jun 2016 → …

Conference

ConferenceSignals and Systems Conference (ISSC), 2016 27th Irish
Period21/06/16 → …

Keywords

  • flood event image recognition
  • social media analysis
  • multimodal data fusion
  • emergency management

Fingerprint Dive into the research topics of 'Integration of text and image analysis for flood event image recognition'. Together they form a unique fingerprint.

Cite this