Large vector spatial data storage and query processing using Clickhouse

Shuaijun Chen, Zhibao Wang, Lu Bai, Kunyi Liu, Juntao Gao, Man Zhao, Maurice Mulvenna

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

126 Downloads (Pure)

Abstract

The exponential growth of geospatial data resulting from the development of earth observation technology has created significant challenges for traditional relational databases. While NoSQL databases based on distributed file systems can handle massive data storage, they often struggle to cope with real-time query. Column-storage databases, on other hand, are highly effective at both
storage and query processing for large-scale datasets. In this paper, we propose a spatial version of ClickHouse that leverages R-Tree indexing to enable efficient storage and real-time analysis of massive remote sensing data. ClickHouse is a column-oriented, open source database management system designed for handling large-scale datasets. By integrating R-Tree indexing, we have created a
highly efficient system for storing and querying geospatial data. To evaluate the performance of our system, we compare it with HBase, a popular distributed, NoSQL database system. Our experimental results show that ClickHouse outperforms HBase in handling spatial data queries, with a response time approximately three times faster than HBase. We attribute this performance gain
to the highly efficient R-Tree indexing used in ClickHouse, which allows for fast spatial data query.
Original languageEnglish
Title of host publicationThe International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Subtitle of host publication39th International Symposium on Remote Sensing of Environment (ISRSE-39) “From Human Needs to SDGs”
Place of PublicationAntalya, Türkiye
Pages65-72
Number of pages8
Volume48
EditionM-1-2023
DOIs
Publication statusPublished (in print/issue) - 25 Apr 2023
Event39th International Symposium on Remote Sensing of Environment - Antalya, Turkey
Duration: 24 Apr 202328 Apr 2023
Conference number: 39
https://isprs-archives.copernicus.org/articles/XLVIII-M-1-2023/65/2023/

Publication series

NameInternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives
PublisherInternational Society for Photogrammetry and Remote Sensing
ISSN (Print)1682-1750

Conference

Conference39th International Symposium on Remote Sensing of Environment
Abbreviated titleISRSE
Country/TerritoryTurkey
CityAntalya
Period24/04/2328/04/23
Internet address

Bibliographical note

Publisher Copyright:
© 2023 International Society for Photogrammetry and Remote Sensing. All rights reserved.

Keywords

  • ClickHouse
  • vector spatial data
  • query processing
  • HBase
  • remote sensing

Fingerprint

Dive into the research topics of 'Large vector spatial data storage and query processing using Clickhouse'. Together they form a unique fingerprint.

Cite this