Revisiting the Encoding of Satellite Image Time Series

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

12 Downloads (Pure)


Satellite Image Time Series (SITS) representation learning is complex due to high
spatiotemporal resolutions, irregular acquisition times, and intricate spatiotemporal interactions.
These challenges result in specialized neural network architectures tailored
for SITS analysis. The field has witnessed promising results achieved by pioneering researchers,
but transferring the latest advances or established paradigms from Computer
Vision (CV) to SITS is still highly challenging due to the existing suboptimal representation
learning framework. In this paper, we develop a novel perspective of SITS
processing as a direct set prediction problem, inspired by the recent trend in adopting
query-based transformer decoders to streamline the object detection or image segmentation
pipeline. We further propose to decompose the representation learning process of
SITS into three explicit steps: collect–update–distribute, which is computationally efficient
and suits for irregularly-sampled and asynchronous temporal satellite observations.
Facilitated by the unique reformulation, our proposed temporal learning backbone of
SITS, initially pre-trained on the resource efficient pixel-set format and then fine-tuned
on the downstream dense prediction tasks, has attained new state-of-the-art (SOTA) results
on the PASTIS benchmark dataset. Specifically, the clear separation between temporal
and spatial components in the semantic/panoptic segmentation pipeline of SITS
makes us leverage the latest advances in CV, such as the universal image segmentation
architecture, resulting in a noticeable 2.5 points increase in mIoU and 8.8 points increase
in PQ, respectively, compared to the best scores reported so far.
Original languageEnglish
Title of host publicationRevisiting the Encoding of Satellite Image Time Series
PublisherBritish Machine Vision Association
Publication statusAccepted/In press - 25 Aug 2023


  • Crop mapping
  • satellite data analysis
  • Computer Vision and Pattern Recognition
  • Time series analysis


Dive into the research topics of 'Revisiting the Encoding of Satellite Image Time Series'. Together they form a unique fingerprint.

Cite this