Skip to content
Clark University
Center for
Geospatial Analytics

Foundation Models for Geospatial Analytics (Collaboration with NASA and IBM)

This project, which has been part of a collaboration with NASA IMPACT and IBM since 2023, is focused on development, application, and benchmarking of geospatial foundation models for Earth Observations. As a member of the project developing Prithvi-EO model, Clark Center for Geospatial team works on evaluation and benchmarking of the model for various downstream applications. In addition, the Clark team is generating a new benchmark to characterize geospatial foundation models accuracy, transparency, and decision-making in different scenarios.

Background

With rapidly expanding satellite missions, and petabyte-scale data available from these missions, the need for efficient data analysis becomes paramount. Geospatial Foundation Models (FMs) are revolutionizing geospatial analytics by harnessing self-supervised techniques on vast unlabeled datasets, such as global satellite imagery. This innovative approach involves training FMs on extensive datasets and fine-tuning them for specific applications with minimal labeled data, proving highly effective in terms of accuracy and training cost.

Dr. Hamed Alemohammad, Director of Clark Center for Geospatial Analytics, in collaboration with NASA and IBM, is pioneering the development of GFMs. To evaluate the value of FMs in Earth Observation (EO) applications, a team from NASA and IBM, in collaboration with academic partners including Clark Center for Geospatial Analytics has developed Prithvi-EO models utilizing Harmonized Landsat Sentinel (HLS) multispectral data. Prithvi-EO-1.0 was the world’s largest geospatial AI model when it was released in August 2023, and Prithvi-EO-2.0 is six-times bigger than its predecessor with 600 million parameters.

foundation model graphic

Impact and Innovation

The latest version of the model, Prithvi-EO-2.0, is trained on 4.2M global time series samples (aka tiles) from NASA’s Harmonized Landsat and Sentinel-2 data archive at 30m resolution, and it incorporates temporal and location embeddings for enhanced performance across various geospatial tasks.

The Prithvi-EO model has demonstrated significant improvements in diverse applications form mapping floods to burn scars, showcasing its potential impact. Its accuracy outperforms six other geospatial foundation models when benchmarked on remote sensing tasks from different domains and resolutions (i.e. from 0.1m to 15m). The results demonstrate the versatility of the model in both classical earth observation and high-resolution applications.

Clark Center for Geospatial Analytics team has implemented multiple downstream applications for cloud gap filling, multi-temporal crop and land cover classification, forest biomass estimation and classification of drivers of deforestation. The cloud gap filling application shows that the Prithvi model can predict multi-spectral cloudy pixels across complex geographies with high accuracy. Fine-tuning Prithvi with varying sample sizes shows that it consistently performs better compared to a baseline generative adversarial network (GAN) model, and it better preserves the relationship between multispectral bands.

XAI Benchmark

AI models learn patterns from the data that are input to them during the training phase. While some of these models have built in constraints based on known physics of the world, majority of the models learn a representation (aka embedding) of the input data without any physical constraint. Hence, there is a need to investigate these models in terms of their learnings and characterize the properties of these representations. The field of explainable (XAI) is defined on research that improves the transparency of AI models and quantifies how these models make decisions.

The Clark Center for Geospatial Analytics is generating an XAI benchmark for GFMs. This benchmark aims to assess whether a GMF learns geospatial properties such as spectral, temporal and spatial characteristics that are embedded in the raw data. Using this benchmark, users can investigate which models are more suitable for their specific downstream application.

Resources